Difference between revisions of "Build FieldWorks"

From LSDevLinux
Jump to: navigation, search
m (To build FieldWorks without running tests: Fix missing piece in the script command)
(Run FieldWorks)
 
(10 intermediate revisions by 2 users not shown)
Line 17: Line 17:
 
For Ubuntu, you may need to enable ''Universe'' and ''Multiverse'' in your software sources (eg via '''Settings ‣ Repositories''' in Synaptic).
 
For Ubuntu, you may need to enable ''Universe'' and ''Multiverse'' in your software sources (eg via '''Settings ‣ Repositories''' in Synaptic).
  
In Ubuntu or Debian, install these dependencies:
+
In Ubuntu or Debian, install dependencies:
 +
 
 +
# Install repository key
 +
$ wget -q -O - <nowiki>http://linux.lsdev.sil.org/downloads/sil-testing.gpg</nowiki> | sudo apt-key add -
 +
 +
# Add SIL testing repository. If using Debian, use something like <nowiki>http://linux.lsdev.sil.org/debian</nowiki> testing main contrib non-free
 +
$ sudo add-apt-repository "deb <nowiki>http://linux.lsdev.sil.org/ubuntu</nowiki> $(lsb_release -sc)              main restricted universe multiverse"
 +
$ sudo add-apt-repository "deb <nowiki>http://linux.lsdev.sil.org/ubuntu</nowiki> $(lsb_release -sc)-experimental main restricted universe multiverse"
 +
$ sudo apt-get update
 +
 
 
  $ sudo apt-get install \
 
  $ sudo apt-get install \
 
   build-essential automake \
 
   build-essential automake \
Line 27: Line 36:
 
   openjdk-6-jre \
 
   openjdk-6-jre \
 
   unzip gawk tofrodos \
 
   unzip gawk tofrodos \
   ttf-mscorefonts-installer ttf-sil-charis \
+
   ttf-mscorefonts-installer fonts-sil-charis \
 
   xvfb xserver-xephyr metacity \
 
   xvfb xserver-xephyr metacity \
   ibus
+
   ibus \
 
+
  geckofx xulrunner-geckofx
Ubuntu 12.04 is missing xulrunner-1.9.2 which we currently require to run FW. However we have built our own package at packages.sil.org in the experimental section.
 
 
 
# Install SIL repository key
 
wget -q -O - <nowiki>http://packages.sil.org/sil.gpg</nowiki> | sudo apt-key add -
 
 
# Add SIL repository and install xulrunner-1.9.2
 
sudo add-apt-repository "deb <nowiki>http://packages.sil.org/ubuntu</nowiki> $(lsb_release -sc)-experimental main"
 
sudo apt-get update
 
sudo apt-get install -y xulrunner-1.9.2
 
  
 
==Register==
 
==Register==
Follow instructions to [https://github.com/sillsdev/FwDocumentation/wiki/Getting-Started-with-FieldWorks-Development#wiki-register Register with services], [https://github.com/sillsdev/FwDocumentation/wiki/Getting-Started-with-FieldWorks-Development#wiki-permissions Request Permissions], and [https://github.com/sillsdev/FwDocumentation/wiki/Getting-Started-with-FieldWorks-Development#wiki-sshkey Upload ssh key]. Do not answer the "Gerrit username" question from the below <code>fwmeta/initrepo</code> script until you have completed these steps.
+
Follow instructions to [https://github.com/sillsdev/FwDocumentation/wiki/Getting-Started-for-Core-Developers#wiki-register Register with services], [https://github.com/sillsdev/FwDocumentation/wiki/Getting-Started-for-Core-Developers#wiki-permissions Request Permissions], and [https://github.com/sillsdev/FwDocumentation/wiki/Getting-Started-for-Core-Developers#wiki-sshkey Upload ssh key]. Do not answer the "Gerrit username" question from the below <code>fwmeta/initrepo</code> script until you have completed these steps.
  
 
==Check out from VCS==
 
==Check out from VCS==
Line 52: Line 52:
 
  fwrepo$ cd fw
 
  fwrepo$ cd fw
 
Your FieldWorks code is in <code>~/fwrepo/fw</code> .
 
Your FieldWorks code is in <code>~/fwrepo/fw</code> .
 +
 +
Install more needed packages:
 +
cd "$HOME/fwrepo/fw/Build/Agent"
 +
./install-deps
  
 
==Mono==
 
==Mono==
Line 130: Line 134:
 
When debugging and developing individual tests, it's not necessary to use a virtual GUI environment. This section gives instructions on how to run specific tests.
 
When debugging and developing individual tests, it's not necessary to use a virtual GUI environment. This section gives instructions on how to run specific tests.
 
===Managed===
 
===Managed===
Unit tests can be run by doing:
+
 
  fwrepo/fw$ (. environ && cd Build && xbuild /t:''project''Tests /property:action=test)
+
Managed unit tests can be run directly from within Monodevelop.
 +
 
 +
Managed unit tests can also be run by doing one of the following:
 +
 
 +
  fwrepo/fw$ Build/run-in-environ mono Bin/NUnit/bin/nunit.exe Output/Debug/''xWorks''Tests.dll
 +
 
 +
fwrepo/fw$ Build/run-in-environ mono --debug Bin/NUnit/bin/nunit-console.exe Output/Debug/''xWorks''Tests.dll
 +
 
 +
fwrepo/fw/Build$ ./build /t:''project''Tests /property:action=test
  
 
where ''project'' is something like <code>LexTextExe</code> or <code>SimpleRootSite</code>.
 
where ''project'' is something like <code>LexTextExe</code> or <code>SimpleRootSite</code>.
  
 
To run a specific test:
 
To run a specific test:
  fwrepo/fw$ export PROJ="''FwCoreDlgs''"
+
  export PROJ="''FwCoreDlgsTests''"
  fwrepo/fw$ export TEST="''SIL.FieldWorks.FwCoreDlgs.CharContextCtrlTests.GetTokenSubstrings_NoValidator''"
+
  export TEST="''SIL.FieldWorks.FwCoreDlgs.CharContextCtrlTests.GetTokenSubstrings_NoValidator''"
  fwrepo/fw$ (. environ && cd Build && xbuild /t:$PROJ)
+
  cd ~/fwrepo/fw/Build && ./build /t:$PROJ
  fwrepo/fw$ (. environ && mono --debug ./Bin/NUnit/bin/nunit-console.exe Output/Debug/${PROJ}Tests.dll -run=$TEST)
+
  cd ~/fwrepo/fw && Build/run-in-environ mono --debug Bin/NUnit/bin/nunit-console.exe Output/Debug/$PROJ.dll -run=$TEST
  
where <code>PROJ</code> is set to the project the test is in and <code>TEST</code> is set to the test to run.
+
where <code>PROJ</code> is set to the project containing the test and <code>TEST</code> is set to the test to run.
  
 
===Native===
 
===Native===
 
Rebuild a library and tests:
 
Rebuild a library and tests:
  fwrepo/fw$ (. environ && cd Build && xbuild /t:views)
+
  fwrepo/fw$ (. environ && cd Build && xbuild /t:testViews)
 
Run all tests in a library:
 
Run all tests in a library:
 
  fwrepo/fw$ (. environ && cd Output/Debug && ./testViews)
 
  fwrepo/fw$ (. environ && cd Output/Debug && ./testViews)
Line 174: Line 186:
  
 
==Run FieldWorks==
 
==Run FieldWorks==
 
fw$ . environ
 
fw$ cd Output/Debug
 
  
 
Run Language Explorer
 
Run Language Explorer
  
Output/Debug$ mono FieldWorks.exe -app Flex
+
  cd ~/fwrepo/fw
 +
  Build/run-in-environ mono Output/Debug/FieldWorks.exe
  
Run Translation Editor
+
View additional FieldWorks logging information by running this after launching FW or opening a FW project:
  
Output/Debug$ mono FieldWorks.exe -app TE
+
  tail -F /tmp/SIL/SIL\ FieldWorks/*
  
 
Running either application will bring up a dialog box so you can navigate to the sample data in <code>DistFiles/ReleaseData</code>.
 
Running either application will bring up a dialog box so you can navigate to the sample data in <code>DistFiles/ReleaseData</code>.

Latest revision as of 13:45, 9 November 2020

These are instructions to build FieldWorks on Linux. To download and run FieldWorks on Linux (rather than compiling it yourself), see FieldWorks Testing Instructions.

For information about building FieldWorks for Mac, see Build FieldWorks for Mac.

To set up a machine to develop FieldWorks, and run a script that does many of the instructions on this page, see Set up new FieldWorks development workstation.

Prerequisites

  • FieldWorks is regularly built and developed on Ubuntu and Debian Linux, and these instructions are written for users of these distributions to build FieldWorks. Ubuntu 12.04 and Debian 7.0 are known to work.
  • A VCS account is only required for write-access to the repository. An anonymous login can be used for read-only access.
  • These instructions are written for users who are reasonably comfortable using the command-line.

Install and build dependencies

For Ubuntu, you may need to enable Universe and Multiverse in your software sources (eg via Settings ‣ Repositories in Synaptic).

In Ubuntu or Debian, install dependencies:

# Install repository key
$ wget -q -O - http://linux.lsdev.sil.org/downloads/sil-testing.gpg | sudo apt-key add -

# Add SIL testing repository. If using Debian, use something like http://linux.lsdev.sil.org/debian testing main contrib non-free
$ sudo add-apt-repository "deb http://linux.lsdev.sil.org/ubuntu $(lsb_release -sc)              main restricted universe multiverse"
$ sudo add-apt-repository "deb http://linux.lsdev.sil.org/ubuntu $(lsb_release -sc)-experimental main restricted universe multiverse"
$ sudo apt-get update
$ sudo apt-get install \
 build-essential automake \
 uuid-dev libenchant-dev xutils-dev happycoders-libsocket \
 libboost-dev libboost-test-dev libmagick++-dev libxklavier-dev \
 libgraphite-dev unixodbc-dev libgtkmm-2.4-dev \
 mono-gmcs mono-devel gtk-sharp2 \
 git git-flow git-gui \
 openjdk-6-jre \
 unzip gawk tofrodos \
 ttf-mscorefonts-installer fonts-sil-charis \
 xvfb xserver-xephyr metacity \
 ibus \
 geckofx xulrunner-geckofx

Register

Follow instructions to Register with services, Request Permissions, and Upload ssh key. Do not answer the "Gerrit username" question from the below fwmeta/initrepo script until you have completed these steps.

Check out from VCS

See further explanation at Getting the git repos. You need the following repos: FieldWorks FwSampleProjects mono mono-basic gtk-sharp libgdiplus libcom

$ cd ~
$ git clone git://github.com/sillsdev/fwmeta.git fwrepo
$ cd fwrepo
fwrepo$ fwmeta/initrepo
fwrepo$ cd fw

Your FieldWorks code is in ~/fwrepo/fw .

Install more needed packages:

cd "$HOME/fwrepo/fw/Build/Agent"
./install-deps

Mono

Currently, to build and run FieldWorks requires a special version of mono. So you will need to build and install mono from source.

Build

Notes:

  1. Please keep in mind that Linux is case sensitive.
  2. It's safer to copy-paste commands than try to retype them.
  3. All of the following commands assume your current directory is fwrepo/fw.

To build FieldWorks without running tests

Ensure you are in the FieldWorks repo directory:

$ cd ~/fwrepo/fw

Run the build:

fwrepo/fw$ (. environ && cd Build && xbuild /t:refreshTargets && xbuild)

If you wish to save a copy of the build output, eg to send to someone else, you can pipe the output through tee:

fwrepo/fw$ (. environ && cd Build && xbuild /t:refreshTargets && xbuild) |& tee build.out

However, xbuild helpfully summarizes any errors at the end of its run, so you can usually just copy the relevant lines directly from the terminal window.

Piping through tee loses the nice text coloring that xbuild provides, so if you want to retain this and still have a copy of the output, you can use the script utility:

fwrepo/fw$ script -ec '. environ && cd Build && xbuild /t:refreshTargets && xbuild'

The output will be saved in a file called typescript. (See man script for how to change this.)

Since script saves an exact copy of the output, including the control sequences used to produce colored and bolded text, this style of output is not so helpful for sending to someone else. For this purpose, tee is better.

To build FieldWorks with tests

If you are doing actual development, ie making changes to the code, it's highly recommended that you run the full suite of unit tests as these will catch a wide range of possible errors. However, this requires a more complex method for running the build since many of the tests perform GUI operations. Without extra set-up, not only is this extremely inconvenient to your use of the computer while the tests are running, it also produces incorrect results due to minor differences in GUI environments.

The set-up involves running the build process in a virtual GUI environment created using the Xephyr utility:

(Note: Don't forget the & symbol at the end of some lines.)

fw$ Xephyr :2 -screen 1024x768 &    # Then press Control-C to make it clear that you still have a prompt
fw$ export DISPLAY=:2 AssertUiEnabled=false
fw$ metacity &
fw$ . environ
fw$ cd Build
fw/Build$ xbuild /t:refreshTargets
fw/Build$ xbuild /property:action=test

(Without setting AssertUiEnabled to false when running the tests, if an assertion fails the message box will pause the build waiting for the user.)

On subsequent builds, only the last command (xbuild /property:action=test) needs to be re-run. However, if you closed the Xephyr window, open a new terminal window/tab and re-run all the commands.

For instructions on how to run specific tests for the purpose of debugging them, see the section on "Unit Tests" below.

To see a test report after a build, open Output/Debug/NUnit-report.html in a browser:

fwrepo/fw$ firefox Output/Debug/NUnit-report.html

(Note: this is generated by the mkall target and so is available only after a complete build.)

To generate a report for a specific test or project, use the NUnitReport.exe utility located in the Build directory.

Other build options

Extra parameters can be given to xbuild to customize the build, for example to build just a part of FieldWorks, or to run a subset of the tests.

The action parameter (specified with /property:action=) can be:

build – just build (the default)
test – run tests as well as build
clean – remove the build products
rebuild – clean followed by build

The target parameter (specified with /t:) can be any target defined in the build system, such as allCsharp, allCpp or XMLUtils. For some specific examples, see below.

Unit tests

When debugging and developing individual tests, it's not necessary to use a virtual GUI environment. This section gives instructions on how to run specific tests.

Managed

Managed unit tests can be run directly from within Monodevelop.

Managed unit tests can also be run by doing one of the following:

fwrepo/fw$ Build/run-in-environ mono Bin/NUnit/bin/nunit.exe Output/Debug/xWorksTests.dll
fwrepo/fw$ Build/run-in-environ mono --debug Bin/NUnit/bin/nunit-console.exe Output/Debug/xWorksTests.dll
fwrepo/fw/Build$ ./build /t:projectTests /property:action=test

where project is something like LexTextExe or SimpleRootSite.

To run a specific test:

export PROJ="FwCoreDlgsTests"
export TEST="SIL.FieldWorks.FwCoreDlgs.CharContextCtrlTests.GetTokenSubstrings_NoValidator"
cd ~/fwrepo/fw/Build && ./build /t:$PROJ
cd ~/fwrepo/fw && Build/run-in-environ mono --debug Bin/NUnit/bin/nunit-console.exe Output/Debug/$PROJ.dll -run=$TEST

where PROJ is set to the project containing the test and TEST is set to the test to run.

Native

Rebuild a library and tests:

fwrepo/fw$ (. environ && cd Build && xbuild /t:testViews)

Run all tests in a library:

fwrepo/fw$ (. environ && cd Output/Debug && ./testViews)

Run just one test in a library:

fwrepo/fw$ (. environ && cd Output/Debug && ./testViews Notifier.DeleteAllParasWithDivs)

To see more information:

fwrepo/fw$ (. environ && cd Output/Debug && ./testViews -v)

Debugging native code called from a managed unit test

In Linux

fwrepo/fw$ . environ
fwrepo/fw$ cd Output/Debug
fwrepo/fw/Output/Debug$ gdb -silent --args /usr/local/bin/mono /usr/local/lib/mono/2.0/nunit-console.exe \
                                               FooTests.dll --run=SIL.FieldWorks.Foo.Test
(gdb) b Program.cpp:1234
(gdb) r

In Windows

Output\Debug>..\..\Bin\NUnit\bin\nunit.exe FooTests.dll
  • Run the unit test to make sure it behaves as expected.
  • Open a new copy of Visual Studio. Choose Debug > Attach to Process. Click Select, click Debug these code types, and select Managed and Native. Click OK. In Available Processes, click nunit.exe and click Attach.
  • Choose File > Open > File to open Program.cpp, and set a breakpoint on the appropriate line.
  • Run the unit test again.

Profiling

fwrepo/fw/Output/Debug$ mono --profile --debug ../../Bin/NUnit/bin/nunit-console.exe \
  AssemblyContainingTest.dll -run=SIL.FieldWorks.FooNamespace.BazClassTests.BlahMethod_toTest

More information about mono profiling can be found here.

Run FieldWorks

Run Language Explorer

 cd ~/fwrepo/fw
 Build/run-in-environ mono Output/Debug/FieldWorks.exe

View additional FieldWorks logging information by running this after launching FW or opening a FW project:

 tail -F /tmp/SIL/SIL\ FieldWorks/*

Running either application will bring up a dialog box so you can navigate to the sample data in DistFiles/ReleaseData.

If you want a blue sidebar in TE then run this, but be aware this is experimental:

MONO_VISUAL_STYLES=gtkplus MONO_VISUAL_STYLE_COLOR_SCHEME=NormalColor MONO_THEME=VisualStyles mono FieldWorks.exe -app Flex

Build script

You may find it helpful to sometimes use the Continuous Integration server's build script. You may want to peek at the script first to make sure it will do what you want. To build FieldWorks using the CI's build script:

fwrepo/fw$ ./agent-build-fw.sh

Cleanup

Running FieldWorks creates some data and config files that a developer may want to clean up to get back to a clean state. Those files include:

~/.local/share/SIL*
~/.mono/registry/CurrentUser/software/{santafe,scrchecks,sil,sil\ international}
~/.mono/mwf_config
~/fwrepo/fw/Output_$(arch)/registry
/tmp/SIL

Editing code

You may find it helpful to Install and use Monodevelop to edit the FieldWorks code.

Troubleshooting

  • If building FieldWorks produces the error message "TLB files are out of date. Please run 'nant idl'.", it may be that the infrastructure isn't updating some files, but it may be that your idhfiles file may need to be regenerated. You can try to fix the error by doing the following:
fwrepo/fw$ mv Output_$(uname -m)/idhfiles{,-outOfWay}
fwrepo/fw$ cd Bld && (pushd .. && . environ && popd && DISPLAY=:0 NAnt.exe checkTLBsUpToDate)