Model Guide

This user guide provided more in depth information about the pipelines of training the model and for parsing a sentence.

Training Pipeline

The training pipeline is shown below.

Load data

The data is loaded from an sqlite3 database of labelled sentences.

See Training data for more information.

Normalise

The input sentences are normalised to clean up particular sentence features into a standard format. The sentence is then tokenised.

See Normalisation for more information.

Extract features

The features for each token are extracted. These features are used to train the model or, once the model has been trained, label each token.

See Extracting features for more information.

Train

The Conditional Random Fields model is trained on 75% of the training data.

Evaluate

The remaining 25% of the training data is used to evaluate the performance of the model on data the model has not encountered before.

See Training the model for more information.

Parsing Pipeline

The parsing pipeline is shown below.

The Normalise and Extract features steps are the same as above.

Label

The features for each token in the sentence are fed into the CRF model which returns a label and the confidence for the label for each token in the sentence.

See Using the model for more information.

Post-process

The token labels go through a post-processing step to build the object that is output from the parse_ingredient function.

See Post-processing for more information.