Support for users of CCG software closed 7-27-20

[Nicholas Rizzolo <rizzolo AT gmail.com>]

Hi Raghavan,

If I may add a little to what Mark said, the FEX feature extraction
language was designed specifically with the NLP domain and the SNoW
machine learning software in mind.  Fex takes either plain text or a
column format as input and produces lists of integers representing
examples as output; exactly the format that snow expects as input.  To
create a program that uses a snow-learned classifier internally, most
developers write scripts that wrap fex and snow, running each as a
separate process during both training and testing.  The car detection
software and the SRL package on our website are examples of this
design paradigm (although a LBJ version of the SRL package is under
development).  We can also provide our snow-based POS tagger and
shallow parser if you are interested; however, these packages are no
longer supported, and they don't perform as well as the new LBJ
versions.

Our current POS tagger, chunker, and state-of-the-art NER and
coreference packages have all been designed with LBJ.  LBJ integrates
and generalizes the functionality of fex and snow in a single
environment.  An LBJ source file (written by you, the programmer)
specifies how to do feature extraction over arbitrary Java objects of
your design, which learning algorithm to apply, which parameters to
tune, etc.  The LBJ compiler automates the feature extraction and
learning processes, managing all the intermediate representations for
you.

An LBJ source file also acts as a Makefile of sorts.  When you make a
change to your LBJ source file, LBJ knows which operations need to be
repeated.  For example, when you change the code in a hard-coded
classifier, only those learned classifiers that use it as a feature
will be retrained.  When you change only a learning algorithm
parameter, LBJ skips feature extraction and goes straight to learning.

Even if you don't want to learn your own classifier, LBJ will still
make it easy to integrate any learned classifier into your Java
program.  When you write that program, you call the classifier by
calling a single method generated by LBJ, giving one of the
classifier's example objects (the same as it was trained on) as input.
 The code generated by LBJ and the LBJ runtime library take care of
the details of performing feature extraction and loading and
evaluating the learned model, returning to you only the information
you are interested in (e.g., a POS tag).  The LBJ-based distributions
mentioned above all come with simple programs that take plain text as
input and evaluate the learned classifiers on it; these programs were
written in exactly this way.

See LBJ's webpage for more details and examples:
http://cogcomp.cs.illinois.edu/page/software_view/11
and let me know if you have any questions.

 - Nick


On Tue, Apr 12, 2011 at 9:26 AM, Mark Sammons 
<mssammon AT illinois.edu>
 wrote:
> SNoW and FEX are completely independent of LBJ; you can read
> the blurb and the documentation for LBJ to see for yourself the many
> ways it can help developers to cleanly integrate machine learning
> directly into their software.  FEX and SNoW were developed to
> work together but are harder to integrate directly into software.
>
> Regards,
>
> Mark
>
> ---- Original message ----
>>Date: Tue, 12 Apr 2011 09:40:31 -0400 (EDT)
>>From: 
>>raghavanks AT aim.com
>>Subject: [Illinois-ml-nlp-users] relationship between various software 
>>packages
>>To: 
>>illinois-ml-nlp-users AT cs.uiuc.edu
>>
>>   Hi,
>>            I am trying to understand the relationship
>>   between the various packages listed here :
>>   http://cogcomp.cs.illinois.edu/page/software . Are
>>   the following packaged meant to work with each other
>>   in some way or are they completely independent ?
>>   SNoW Learning Architecture,  Learning Based Java ,
>>   Feature Extraction Language (FEX) .  I do see that
>>   the POS Tagger, Chunker are based on LBJ. Are these
>>   meant to work with sNoW Learning Architecture or are
>>   there equivalents available inside sNoW ?
>>
>>   Thanks,
>>   Raghavan
>>________________
>>_______________________________________________
>>illinois-ml-nlp-users mailing list
>>illinois-ml-nlp-users AT cs.uiuc.edu
>>http://lists.cs.uiuc.edu/mailman/listinfo/illinois-ml-nlp-users
> _______________________________________________
> illinois-ml-nlp-users mailing list
> illinois-ml-nlp-users AT cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/illinois-ml-nlp-users
>