Making a release of hfst (same as any FLOSS project, basically)

When a HFST source tree is considered stable and usable enough to inflict it upon wider audiences than the version control tree (which is meant for early adopters and testers who know things are segfaulting buggy concoctions over there), a certain number of upkeeping rituals needs to be done.

Firstly ensuring that everything is in place and seemingly workable:

$ svn co https://svn.hfst.sourceforge.net/svnroot/hfst/trunk/hfst??? \
hfst???-releasetest
[...]
$ cd hfst???-releasetest
$ autoreconf -i
$ ./configure
$ make distcheck
$ scp hfst???.tar.gz another-machine:.
$ ssh another-machine
$ tar zxvvf hfst???.tar.gz
$ cd hfst???
$ ./configure
$ make
$ make check
$ sudo make install

Assuming everything is in place, update NEWS and ChangeLog with info about the purpose of this release. The NEWS file shall contain human-readable summary of the major changes since last release (if there are no major changes, do not bother yourself with releasing ;-). The ChangeLog is nearly machine-readable file of all changes in all of the files in the source tree. Use scripts like svn2cl to generate the skeleton and sed to change nicknames into realnames and such. Commit these to trunk.

Merge changes to stable tree. This will usually work just nicely since stable releases will not have changes that cause conflicts. If there are changes to API or backwards incompatible changes you should not be making stable release in the first place.

$ svn co https://hfst.svn.sourceforge.net/svnroot/hfst/releases/stable-x.y \
old-stable
$ cd old-stable
$ svn merge https://hfst.svn.sourceforge.net/svnroot/hfst/trunk/hfst???
[double-check that everything merged nicely]
[assuming it didn't, revert and do a step-by-step svn merge -c]
$ svn commit

Copy the distribution tarball to sourceforge's file release system and tag the version trees:

$ make dist
$ scp hfst???.tar.gz \
username,hfst@frs.sourceforge.net:/home/frs/project/h/hf/hfst/hfst/
$ svn cp https://svn.hfst.sourceforge.net/svnroot/hfst/trunk/hfst??? \
https://svn.hfst.sourceforge.net/svnroot/hfst/tags/release???

Release is done. Now you can play games with sourceforge's web-based admin tools to tweak the details.

Debuging transducers with awk and hfst-tools

Dump transducer to text file:

$ hfst-fst2txt -o omorfi.lexc.hfst.txt -i omorfi.lexc.hfst

Finding paths with label KTN=26:

$ gawk '$3 ~ /=26/' omorfi.lexc.fst.txt
23595   23596   [KTN=26]        @0@
28182   126     [KTN=26]        @0@
32663   363     [KTN=26]        @0@
85125   3       [KTN=26]        @0@

Track leftwards...

$ gawk '$2 = 23595' omorfi.lexc.fst.txt

[wash, rinse,  repeat]

Track rightwards...

$ gawk '$1 = 3' omorfi.lexc.fst.txt

[wash, rinse, repeat]

HFST code fragments

For functions I cannot remember and have to always look up from somewhere.

Printing a key:

  char * name;
  if (weighted)
  {
    HWFST::KeyTable kt = ...;
    HWFST::Key key = ...;
    HWFST::Symbol symbol = HWFST::get_key_symbol(key, kt);
    name = HWFST::get_symbol_name(symbol);
  }
  else if (!weighted)
  {
    HFST::KeyTable kt = ...;
    HFST::Key key = ...;
    HFST::Symbol symbol = HFST::get_key_symbol(key, kt);
    name = HFST::get_symbol_name(symbol);
  }
  printf("%s", name);

Findin a key for a string:

  char *s = ...;
  if (weighted)
  {
    HWFST::KeyTable kt = ...;
    if (!HWFST::is_symbol(s, kt))
    {
      HWFST::Symbol sy = HWFST::define_symbol(s);
      HWFST::Key k = HWFST::get_unused_key(kt);
      HWFST::associate_key(k, kt, sy);
    }
    return HWFST::get_key(HWFST::get_symbol(s, kt));
  }
  else if (!weighted)
  {
    HFST::KeyTable kt = ...;
    if (!HFST::is_symbol(s, kt))
    {
      HFST::Symbol sy = HFST::define_symbol(s);
      HFST::Key k = HFST::get_unused_key(kt);
      HFST::associate_key(k, kt, sy);
    }
    return HFST::get_key(HFST::get_symbol(s, kt));
  }

-- TommiPirinen