Package 'aifeducation'

Description

This base class is used to create and define .AIFE*Transformer-like classes. It serves as a skeleton for a future concrete transformer and cannot be used to create an object of itself (an attempt to call new-method will produce an error).

See p.1 Base Transformer Class in Transformers for Developers for details.

Create

The create-method is a basic algorithm that is used to create a new transformer, but cannot be called directly.

Train

The train-method is a basic algorithm that is used to train and tune the transformer but cannot be called directly.

Concrete transformer implementation

There are already implemented concrete (child) transformers (e.g. BERT, DeBERTa-V2, etc.), to implement a new one see p.4 Implement A Custom Transformer in Transformers for Developers

Public fields

Methods

Public methods

• .AIFEBaseTransformer$new()

• .AIFEBaseTransformer$set_title()

• .AIFEBaseTransformer$set_model_param()

• .AIFEBaseTransformer$set_model_temp()

• .AIFEBaseTransformer$set_SFC_check_max_pos_emb()

• .AIFEBaseTransformer$set_SFC_create_tokenizer_draft()

• .AIFEBaseTransformer$set_SFC_calculate_vocab()

• .AIFEBaseTransformer$set_SFC_save_tokenizer_draft()

• .AIFEBaseTransformer$set_SFC_create_final_tokenizer()

• .AIFEBaseTransformer$set_SFC_create_transformer_model()

• .AIFEBaseTransformer$set_required_SFC()

• .AIFEBaseTransformer$set_SFT_load_existing_model()

• .AIFEBaseTransformer$set_SFT_cuda_empty_cache()

• .AIFEBaseTransformer$set_SFT_create_data_collator()

• .AIFEBaseTransformer$create()

• .AIFEBaseTransformer$train()

• .AIFEBaseTransformer$clone()

Method new()

An object of this class cannot be created. Thus, method's call will produce an error.

Usage

.AIFEBaseTransformer$new()

Returns

This method returns an error.

Method set_title()

Setter for the title. Sets a new value for the title private attribute.

Usage

.AIFEBaseTransformer$set_title(title)

Arguments

Returns

This method returns nothing.

Method set_model_param()

Setter for the parameters. Adds a new parameter and its value to the params list.

Usage

.AIFEBaseTransformer$set_model_param(param_name, param_value)

Arguments

Returns

This method returns nothing.

Method set_model_temp()

Setter for the temporary model's parameters. Adds a new temporary parameter and its value to the temp list.

Usage

.AIFEBaseTransformer$set_model_temp(temp_name, temp_value)

Arguments

Returns

This method returns nothing.

Method set_SFC_check_max_pos_emb()

Setter for the check_max_pos_emb element of the private steps_for_creation list. Sets a new fun function as the check_max_pos_emb step.

Usage

.AIFEBaseTransformer$set_SFC_check_max_pos_emb(fun)

Arguments

Returns

This method returns nothing.

Method set_SFC_create_tokenizer_draft()

Setter for the create_tokenizer_draft element of the private steps_for_creation list. Sets a new fun function as the create_tokenizer_draft step.

Usage

.AIFEBaseTransformer$set_SFC_create_tokenizer_draft(fun)

Arguments

Returns

This method returns nothing.

Method set_SFC_calculate_vocab()

Setter for the calculate_vocab element of the private steps_for_creation list. Sets a new fun function as the calculate_vocab step.

Usage

.AIFEBaseTransformer$set_SFC_calculate_vocab(fun)

Arguments

Returns

This method returns nothing.

Method set_SFC_save_tokenizer_draft()

Setter for the save_tokenizer_draft element of the private steps_for_creation list. Sets a new fun function as the save_tokenizer_draft step.

Usage

.AIFEBaseTransformer$set_SFC_save_tokenizer_draft(fun)

Arguments

Returns

This method returns nothing.

Method set_SFC_create_final_tokenizer()

Setter for the create_final_tokenizer element of the private steps_for_creation list. Sets a new fun function as the create_final_tokenizer step.

Usage

.AIFEBaseTransformer$set_SFC_create_final_tokenizer(fun)

Arguments

Returns

This method returns nothing.

Method set_SFC_create_transformer_model()

Setter for the create_transformer_model element of the private steps_for_creation list. Sets a new fun function as the create_transformer_model step.

Usage

.AIFEBaseTransformer$set_SFC_create_transformer_model(fun)

Arguments

Returns

This method returns nothing.

Method set_required_SFC()

Setter for all required elements of the private steps_for_creation list. Executes setters for all required creation steps.

Usage

.AIFEBaseTransformer$set_required_SFC(required_SFC)

Arguments

Returns

This method returns nothing.

Method set_SFT_load_existing_model()

Setter for the load_existing_model element of the private steps_for_training list. Sets a new fun function as the load_existing_model step.

Usage

.AIFEBaseTransformer$set_SFT_load_existing_model(fun)

Arguments

Returns

This method returns nothing.

Method set_SFT_cuda_empty_cache()

Setter for the cuda_empty_cache element of the private steps_for_training list. Sets a new fun function as the cuda_empty_cache step.

Usage

.AIFEBaseTransformer$set_SFT_cuda_empty_cache(fun)

Arguments

Returns

This method returns nothing.

Method set_SFT_create_data_collator()

Setter for the create_data_collator element of the private steps_for_training list. Sets a new fun function as the create_data_collator step. Use this method to make a custom data collator for a transformer.

Usage

.AIFEBaseTransformer$set_SFT_create_data_collator(fun)

Arguments

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the child-transformer architecture and a vocabulary using the python libraries transformers and tokenizers.

This method adds the following parameters to the temp list:

• log_file

• raw_text_dataset

• pt_safe_save

• value_top

• total_top

• message_top

This method uses the following parameters from the temp list:

• log_file

• raw_text_dataset

• tokenizer

Usage

.AIFEBaseTransformer$create(
  ml_framework,
  model_dir,
  text_dataset,
  vocab_size,
  max_position_embeddings,
  hidden_size,
  num_attention_heads,
  intermediate_size,
  hidden_act,
  hidden_dropout_prob,
  attention_probs_dropout_prob,
  sustain_track,
  sustain_iso_code,
  sustain_region,
  sustain_interval,
  trace,
  pytorch_safetensors,
  log_dir,
  log_write_interval
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on BERT architecture with the help of the python libraries transformers, datasets, and tokenizers.

This method adds the following parameters to the temp list:

• log_file

• loss_file

• from_pt

• from_tf

• load_safe

• raw_text_dataset

• pt_safe_save

• value_top

• total_top

• message_top

This method uses the following parameters from the temp list:

• log_file

• raw_text_dataset

• tokenized_dataset

• tokenizer

Usage

.AIFEBaseTransformer$train(
  ml_framework,
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask,
  whole_word,
  val_size,
  n_epoch,
  batch_size,
  chunk_size,
  full_sequences_only,
  min_seq_len,
  learning_rate,
  n_workers,
  multi_process,
  sustain_track,
  sustain_iso_code,
  sustain_region,
  sustain_interval,
  trace,
  keras_trace,
  pytorch_trace,
  pytorch_safetensors,
  log_dir,
  log_write_interval
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFEBaseTransformer$clone(deep = FALSE)

Arguments

References

Hugging Face transformers documantation:

• BERT

• DeBERTa

• Funnel

• Longformer

• RoBERTa

• MPNet

See Also

Other Transformers for developers: .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer, .AIFETrObj

Description

This class has the following methods:

• create: creates a new transformer based on BERT.

• train: trains and fine-tunes a BERT model.

Create

New models can be created using the .AIFEBertTransformer$create method.

Train

To train the model, pass the directory of the model to the method .AIFEBertTransformer$train.

Pre-Trained models that can be fine-tuned using this method are available at https://huggingface.co/.

The model is trained using dynamic masking, as opposed to the original paper, which used static masking.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFEBertTransformer

Methods

Public methods

• .AIFEBertTransformer$new()

• .AIFEBertTransformer$create()

• .AIFEBertTransformer$train()

• .AIFEBertTransformer$clone()

Method new()

Creates a new transformer based on BERT and sets the title.

Usage

.AIFEBertTransformer$new()

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the BERT base architecture and a vocabulary based on WordPiece by using the python libraries transformers and tokenizers.

This method adds the following 'dependent' parameters to the base class's inherited params list:

• vocab_do_lower_case

• num_hidden_layer

Usage

.AIFEBertTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 30522,
  vocab_do_lower_case = FALSE,
  max_position_embeddings = 512,
  hidden_size = 768,
  num_hidden_layer = 12,
  num_attention_heads = 12,
  intermediate_size = 3072,
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  sustain_track = FALSE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on BERT architecture with the help of the python libraries transformers, datasets, and tokenizers.

Usage

.AIFEBertTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  whole_word = TRUE,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.003,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = FALSE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFEBertTransformer$clone(deep = FALSE)

Arguments

Note

This model uses a WordPiece tokenizer like BERT and can be trained with whole word masking. The transformer library may display a warning, which can be ignored.

References

Devlin, J., Chang, M.‑W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In J. Burstein, C. Doran, & T. Solorio (Eds.), Proceedings of the 2019 Conference of the North (pp. 4171–4186). Association for Computational Linguistics. doi:10.18653/v1/N19-1423

Hugging Face documentation

• https://huggingface.co/docs/transformers/model_doc/bert

• https://huggingface.co/docs/transformers/model_doc/bert#transformers.BertForMaskedLM

• https://huggingface.co/docs/transformers/model_doc/bert#transformers.TFBertForMaskedLM

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer, .AIFETrObj

Description

This class has the following methods:

• create: creates a new transformer based on DeBERTa-V2.

• train: trains and fine-tunes a DeBERTa-V2 model.

Create

New models can be created using the .AIFEDebertaTransformer$create method.

Train

To train the model, pass the directory of the model to the method .AIFEDebertaTransformer$train.

Pre-Trained models which can be fine-tuned with this function are available at https://huggingface.co/.

Training of this model makes use of dynamic masking.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFEDebertaTransformer

Methods

Public methods

• .AIFEDebertaTransformer$new()

• .AIFEDebertaTransformer$create()

• .AIFEDebertaTransformer$train()

• .AIFEDebertaTransformer$clone()

Method new()

Creates a new transformer based on DeBERTa-V2 and sets the title.

Usage

.AIFEDebertaTransformer$new()

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the DeBERTa-V2 base architecture and a vocabulary based on the SentencePiece tokenizer using the python transformers and tokenizers libraries.

This method adds the following 'dependent' parameters to the base class's inherited params list:

• vocab_do_lower_case

• num_hidden_layer

Usage

.AIFEDebertaTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 128100,
  vocab_do_lower_case = FALSE,
  max_position_embeddings = 512,
  hidden_size = 1536,
  num_hidden_layer = 24,
  num_attention_heads = 24,
  intermediate_size = 6144,
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on DeBERTa-V2 architecture with the help of the python libraries transformers, datasets, and tokenizers.

Usage

.AIFEDebertaTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  whole_word = TRUE,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.03,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFEDebertaTransformer$clone(deep = FALSE)

Arguments

Note

For this model a WordPiece tokenizer is created. The standard implementation of DeBERTa version 2 from HuggingFace uses a SentencePiece tokenizer. Thus, please use AutoTokenizer from the transformers library to work with this model.

References

He, P., Liu, X., Gao, J. & Chen, W. (2020). DeBERTa: Decoding-enhanced BERT with Disentangled Attention. doi:10.48550/arXiv.2006.03654

Hugging Face documentatio

• https://huggingface.co/docs/transformers/model_doc/deberta-v2

• https://huggingface.co/docs/transformers/model_doc/deberta-v2#transformers.DebertaV2ForMaskedLM

• https://huggingface.co/docs/transformers/model_doc/deberta-v2#transformers.TFDebertaV2ForMaskedLM

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer, .AIFETrObj

Description

This class has the following methods:

• create: creates a new transformer based on Funnel.

• train: trains and fine-tunes a Funnel model.

Create

New models can be created using the .AIFEFunnelTransformer$create method.

Model is created with separete_cls = TRUE, truncate_seq = TRUE, and pool_q_only = TRUE.

Train

To train the model, pass the directory of the model to the method .AIFEFunnelTransformer$train.

Pre-Trained models which can be fine-tuned with this function are available at https://huggingface.co/.

Training of the model makes use of dynamic masking.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFEFunnelTransformer

Methods

Public methods

• .AIFEFunnelTransformer$new()

• .AIFEFunnelTransformer$create()

• .AIFEFunnelTransformer$train()

• .AIFEFunnelTransformer$clone()

Method new()

Creates a new transformer based on Funnel and sets the title.

Usage

.AIFEFunnelTransformer$new()

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the Funnel transformer base architecture and a vocabulary based on WordPiece using the python transformers and tokenizers libraries.

This method adds the following 'dependent' parameters to the base class's inherited params list:

• vocab_do_lower_case

• target_hidden_size

• block_sizes

• num_decoder_layers

• pooling_type

• activation_dropout

Usage

.AIFEFunnelTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 30522,
  vocab_do_lower_case = FALSE,
  max_position_embeddings = 512,
  hidden_size = 768,
  target_hidden_size = 64,
  block_sizes = c(4, 4, 4),
  num_attention_heads = 12,
  intermediate_size = 3072,
  num_decoder_layers = 2,
  pooling_type = "mean",
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  activation_dropout = 0,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on Funnel Transformer architecture with the help of the python libraries transformers, datasets, and tokenizers.

Usage

.AIFEFunnelTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  whole_word = TRUE,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.003,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFEFunnelTransformer$clone(deep = FALSE)

Arguments

Note

The model uses a configuration with truncate_seq = TRUE to avoid implementation problems with tensorflow.

This model uses a WordPiece tokenizer like BERT and can be trained with whole word masking. The transformer library may display a warning, which can be ignored.

References

Dai, Z., Lai, G., Yang, Y. & Le, Q. V. (2020). Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing. doi:10.48550/arXiv.2006.03236

Hugging Face documentation

• https://huggingface.co/docs/transformers/model_doc/funnel#funnel-transformer

• https://huggingface.co/docs/transformers/model_doc/funnel#transformers.FunnelModel

• https://huggingface.co/docs/transformers/model_doc/funnel#transformers.TFFunnelModel

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer, .AIFETrObj

Description

This class has the following methods:

• create: creates a new transformer based on Longformer.

• train: trains and fine-tunes a Longformer model.

Create

New models can be created using the .AIFELongformerTransformer$create method.

Train

To train the model, pass the directory of the model to the method .AIFELongformerTransformer$train.

Pre-Trained models which can be fine-tuned with this function are available at https://huggingface.co/.

Training of this model makes use of dynamic masking.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFELongformerTransformer

Methods

Public methods

• .AIFELongformerTransformer$new()

• .AIFELongformerTransformer$create()

• .AIFELongformerTransformer$train()

• .AIFELongformerTransformer$clone()

Method new()

Creates a new transformer based on Longformer and sets the title.

Usage

.AIFELongformerTransformer$new()

Returns

This method returns nothing

Method create()

This method creates a transformer configuration based on the Longformer base architecture and a vocabulary based on ⁠Byte-Pair Encoding⁠ (BPE) tokenizer using the python transformers and tokenizers libraries.

This method adds the following 'dependent' parameters to the base class's inherited params list:

• add_prefix_space

• trim_offsets

• num_hidden_layer

• attention_window

Usage

.AIFELongformerTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 30522,
  add_prefix_space = FALSE,
  trim_offsets = TRUE,
  max_position_embeddings = 512,
  hidden_size = 768,
  num_hidden_layer = 12,
  num_attention_heads = 12,
  intermediate_size = 3072,
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  attention_window = 512,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on Longformer Transformer architecture with the help of the python libraries transformers, datasets, and tokenizers.

Usage

.AIFELongformerTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.03,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFELongformerTransformer$clone(deep = FALSE)

Arguments

References

Beltagy, I., Peters, M. E., & Cohan, A. (2020). Longformer: The Long-Document Transformer. doi:10.48550/arXiv.2004.05150

Hugging Face Documentation

• https://huggingface.co/docs/transformers/model_doc/longformer

• https://huggingface.co/docs/transformers/model_doc/longformer#transformers.LongformerModel

• https://huggingface.co/docs/transformers/model_doc/longformer#transformers.TFLongformerModel

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer, .AIFETrObj

Description

This class has the following methods:

• create: creates a new transformer based on MPNet.

• train: trains and fine-tunes a MPNet model.

Create

New models can be created using the .AIFEMpnetTransformer$create method.

Train

To train the model, pass the directory of the model to the method .AIFEMpnetTransformer$train.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFEMpnetTransformer

Public fields

Methods

Public methods

• .AIFEMpnetTransformer$new()

• .AIFEMpnetTransformer$create()

• .AIFEMpnetTransformer$train()

• .AIFEMpnetTransformer$clone()

Method new()

Creates a new transformer based on MPNet and sets the title.

Usage

.AIFEMpnetTransformer$new()

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the MPNet base architecture.

This method adds the following 'dependent' parameters to the base class's inherited params list:

• vocab_do_lower_case

• num_hidden_layer

Usage

.AIFEMpnetTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 30522,
  vocab_do_lower_case = FALSE,
  max_position_embeddings = 512,
  hidden_size = 768,
  num_hidden_layer = 12,
  num_attention_heads = 12,
  intermediate_size = 3072,
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  sustain_track = FALSE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on MPNet architecture with the help of the python libraries transformers, datasets, and tokenizers.

This method adds the following 'dependent' parameter to the base class's inherited params list:

• p_perm

Usage

.AIFEMpnetTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  p_perm = 0.15,
  whole_word = TRUE,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.003,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = FALSE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFEMpnetTransformer$clone(deep = FALSE)

Arguments

Note

Using this class with tensorflow is not supported. Supported framework is pytorch.

References

Song,K., Tan, X., Qin, T., Lu, J. & Liu, T.-Y. (2020). MPNet: Masked and Permuted Pre-training for Language Understanding. doi:10.48550/arXiv.2004.09297

Hugging Face documentation

• https://huggingface.co/docs/transformers/model_doc/mpnet

• https://huggingface.co/docs/transformers/model_doc/mpnet#transformers.MPNetForMaskedLM

• https://huggingface.co/docs/transformers/model_doc/mpnet#transformers.TFMPNetForMaskedLM

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFERobertaTransformer, .AIFETrObj

Description

This class has the following methods:

• create: creates a new transformer based on RoBERTa.

• train: trains and fine-tunes a RoBERTa model.

Create

New models can be created using the .AIFERobertaTransformer$create method.

Train

To train the model, pass the directory of the model to the method .AIFERobertaTransformer$train.

Pre-Trained models which can be fine-tuned with this function are available at https://huggingface.co/.

Training of this model makes use of dynamic masking.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFERobertaTransformer

Methods

Public methods

• .AIFERobertaTransformer$new()

• .AIFERobertaTransformer$create()

• .AIFERobertaTransformer$train()

• .AIFERobertaTransformer$clone()

Method new()

Creates a new transformer based on RoBERTa and sets the title.

Usage

.AIFERobertaTransformer$new()

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the RoBERTa base architecture and a vocabulary based on ⁠Byte-Pair Encoding⁠ (BPE) tokenizer using the python transformers and tokenizers libraries.

This method adds the following 'dependent' parameters to the base class' inherited params list:

• add_prefix_space

• trim_offsets

• num_hidden_layer

Usage

.AIFERobertaTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 30522,
  add_prefix_space = FALSE,
  trim_offsets = TRUE,
  max_position_embeddings = 512,
  hidden_size = 768,
  num_hidden_layer = 12,
  num_attention_heads = 12,
  intermediate_size = 3072,
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on RoBERTa Transformer architecture with the help of the python libraries transformers, datasets, and tokenizers.

Usage

.AIFERobertaTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.03,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFERobertaTransformer$clone(deep = FALSE)

Arguments

References

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. doi:10.48550/arXiv.1907.11692

Hugging Face Documentation

• https://huggingface.co/docs/transformers/model_doc/roberta

• https://huggingface.co/docs/transformers/model_doc/roberta#transformers.RobertaModel

• https://huggingface.co/docs/transformers/model_doc/roberta#transformers.TFRobertaModel

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFETrObj

Description

Object for creating the transformers with different types. See AIFETransformerMaker class for details.

Usage

aife_transformer_maker

Format

An object of class AIFETransformerMaker (inherits from R6) of length 3.

See Also

Other Transformer: AIFETrType, AIFETransformerMaker

Examples

# Use 'make' method of the 'aifeducation::aife_transformer_maker' object
# Pass string with the type of transformers
# Allowed types are "bert", "deberta_v2", "funnel", etc. See aifeducation::AIFETrType list
my_bert <- aife_transformer_maker$make("bert")

# Or use elements of the 'aifeducation::AIFETrType' list
my_longformer <- aife_transformer_maker$make(AIFETrType$longformer)

# Run 'create' or 'train' methods of the transformer in order to create a
# new transformer or train the newly created one, respectively
# my_bert$create(...)
# my_bert$train(...)

# my_longformer$create(...)
# my_longformer$train(...)

Description

Abstract class for all models that do not rely on the python library 'transformers'.

Value

Objects of this containing fields and methods used in several other classes in 'ai for education'. This class is not designed for a direct application and should only be used by developers.

Public fields

Methods

Public methods

• AIFEBaseModel$get_model_info()

• AIFEBaseModel$get_text_embedding_model()

• AIFEBaseModel$set_publication_info()

• AIFEBaseModel$get_publication_info()

• AIFEBaseModel$set_model_license()

• AIFEBaseModel$get_model_license()

• AIFEBaseModel$set_documentation_license()

• AIFEBaseModel$get_documentation_license()

• AIFEBaseModel$set_model_description()

• AIFEBaseModel$get_model_description()

• AIFEBaseModel$save()

• AIFEBaseModel$load()

• AIFEBaseModel$get_package_versions()

• AIFEBaseModel$get_sustainability_data()

• AIFEBaseModel$get_ml_framework()

• AIFEBaseModel$get_text_embedding_model_name()

• AIFEBaseModel$check_embedding_model()

• AIFEBaseModel$count_parameter()

• AIFEBaseModel$is_configured()

• AIFEBaseModel$get_private()

• AIFEBaseModel$get_all_fields()

• AIFEBaseModel$clone()

Method get_model_info()

Method for requesting the model information.

Usage

AIFEBaseModel$get_model_info()

Returns

list of all relevant model information.

Method get_text_embedding_model()

Method for requesting the text embedding model information.

Usage

AIFEBaseModel$get_text_embedding_model()

Returns

list of all relevant model information on the text embedding model underlying the model.

Method set_publication_info()

Method for setting publication information of the model.

Usage

AIFEBaseModel$set_publication_info(authors, citation, url = NULL)

Arguments

Returns

Function does not return a value. It is used for setting the private members for publication information.

Method get_publication_info()

Method for requesting the bibliographic information of the model.

Usage

AIFEBaseModel$get_publication_info()

Returns

list with all saved bibliographic information.

Method set_model_license()

Method for setting the license of the model.

Usage

AIFEBaseModel$set_model_license(license = "CC BY")

Arguments

Returns

Function does not return a value. It is used for setting the private member for the software license of the model.

Method get_model_license()

Method for getting the license of the model.

Usage

AIFEBaseModel$get_model_license()

Arguments

Returns

string representing the license for the model.

Method set_documentation_license()

Method for setting the license of the model's documentation.

Usage

AIFEBaseModel$set_documentation_license(license = "CC BY")

Arguments

Returns

Function does not return a value. It is used for setting the private member for the documentation license of the model.

Method get_documentation_license()

Method for getting the license of the model's documentation.

Usage

AIFEBaseModel$get_documentation_license()

Arguments

Returns

Returns the license as a string.

Method set_model_description()

Method for setting a description of the model.

Usage

AIFEBaseModel$set_model_description(
  eng = NULL,
  native = NULL,
  abstract_eng = NULL,
  abstract_native = NULL,
  keywords_eng = NULL,
  keywords_native = NULL
)

Arguments

Returns

Function does not return a value. It is used for setting the private members for the description of the model.

Method get_model_description()

Method for requesting the model description.

Usage

AIFEBaseModel$get_model_description()

Returns

list with the description of the classifier in English and the native language.

Method save()

Method for saving a model.

Usage

AIFEBaseModel$save(dir_path, folder_name)

Arguments

Returns

Function does not return a value. It saves the model to disk.

Method load()

Method for importing a model.

Usage

AIFEBaseModel$load(dir_path)

Arguments

Returns

Function does not return a value. It is used to load the weights of a model.

Method get_package_versions()

Method for requesting a summary of the R and python packages' versions used for creating the model.

Usage

AIFEBaseModel$get_package_versions()

Returns

Returns a list containing the versions of the relevant R and python packages.

Method get_sustainability_data()

Method for requesting a summary of tracked energy consumption during training and an estimate of the resulting CO2 equivalents in kg.

Usage

AIFEBaseModel$get_sustainability_data()

Returns

Returns a list containing the tracked energy consumption, CO2 equivalents in kg, information on the tracker used, and technical information on the training infrastructure.

Method get_ml_framework()

Method for requesting the machine learning framework used for the model.

Usage

AIFEBaseModel$get_ml_framework()

Returns

Returns a string describing the machine learning framework used for the classifier.

Method get_text_embedding_model_name()

Method for requesting the name (unique id) of the underlying text embedding model.

Usage

AIFEBaseModel$get_text_embedding_model_name()

Returns

Returns a string describing name of the text embedding model.

Method check_embedding_model()

Method for checking if the provided text embeddings are created with the same TextEmbeddingModel as the model.

Usage

AIFEBaseModel$check_embedding_model(text_embeddings)

Arguments

Returns

TRUE if the underlying TextEmbeddingModel are the same. FALSE if the models differ.

Method count_parameter()

Method for counting the trainable parameters of a model.

Usage

AIFEBaseModel$count_parameter()

Returns

Returns the number of trainable parameters of the model.

Method is_configured()

Method for checking if the model was successfully configured. An object can only be used if this value is TRUE.

Usage

AIFEBaseModel$is_configured()

Returns

bool TRUE if the model is fully configured. FALSE if not.

Method get_private()

Method for requesting all private fields and methods. Used for loading and updating an object.

Usage

AIFEBaseModel$get_private()

Returns

Returns a list with all private fields and methods.

Method get_all_fields()

Return all fields.

Usage

AIFEBaseModel$get_all_fields()

Returns

Method returns a list containing all public and private fields of the object.

Method clone()

The objects of this class are cloneable with this method.

Usage

AIFEBaseModel$clone(deep = FALSE)

Arguments

Description

This class was developed to make the creation of transformers easier for users. Pass the transformer's type to the make method and get desired transformer. Now run the create or/and train methods of the new transformer.

The already created aife_transformer_maker object of this class can be used.

See p.3 Transformer Maker in Transformers for Developers for details.

See .AIFEBaseTransformer class for details.

Methods

Public methods

• AIFETransformerMaker$make()

• AIFETransformerMaker$clone()

Method make()

Creates a new transformer with the passed type.

Usage

AIFETransformerMaker$make(type)

Arguments

Returns

If success - a new transformer, otherwise - an error (passed type is invalid).

Method clone()

The objects of this class are cloneable with this method.

Usage

AIFETransformerMaker$clone(deep = FALSE)

Arguments

See Also

Other Transformer: AIFETrType, aife_transformer_maker

Examples

# Create transformer maker
tr_maker <- AIFETransformerMaker$new()

# Use 'make' method of the 'tr_maker' object
# Pass string with the type of transformers
# Allowed types are "bert", "deberta_v2", "funnel", etc. See aifeducation::AIFETrType list
my_bert <- tr_maker$make("bert")

# Or use elements of the 'aifeducation::AIFETrType' list
my_longformer <- tr_maker$make(AIFETrType$longformer)

# Run 'create' or 'train' methods of the transformer in order to create a
# new transformer or train the newly created one, respectively
# my_bert$create(...)
# my_bert$train(...)

# my_longformer$create(...)
# my_longformer$train(...)

Description

This list contains transformer types. Elements of the list can be used in the public make of the AIFETransformerMaker R6 class as input parameter type.

It has the following elements:

• bert = 'bert'

• roberta = 'roberta'

• deberta_v2 = 'deberta_v2'

• funnel = 'funnel'

• longformer = 'longformer'

• mpnet = 'mpnet'

  Elements can be used like AIFETrType$bert, AIFETrType$deberta_v2, AIFETrType$funnel, etc.

Usage

AIFETrType

Format

An object of class list of length 6.

See Also

Other Transformer: AIFETransformerMaker, aife_transformer_maker

Description

Function for getting the number of cores that should be used for parallel processing of tasks. The number of cores is set to 75 % of the available cores. If the environment variable CI is set to "true" or if the process is running on cran 2 is returned.

Usage

auto_n_cores()

Value

Returns int as the number of cores.

See Also

Other Utils: clean_pytorch_log_transformers(), create_config_state(), create_dir(), generate_id(), get_file_extension(), get_py_package_versions(), is.null_or_na(), output_message(), print_message(), run_py_file()

Description

Function for calculating recall, precision, and f1.

Usage

calc_standard_classification_measures(true_values, predicted_values)

Arguments

Value

Returns a matrix which contains the cases categories in the rows and the measures (precision, recall, f1) in the columns.

See Also

Other classifier_utils: get_coder_metrics()

Description

This function checks if all python modules necessary for the package aifeducation to work are available.

Usage

check_aif_py_modules(trace = TRUE, check = "pytorch")

Arguments

Value

The function prints a table with all relevant packages and shows which modules are available or unavailable.

If all relevant modules are available, the functions returns TRUE. In all other cases it returns FALSE

See Also

Other Installation and Configuration: install_aifeducation(), install_py_modules(), set_transformers_logger()

Description

Function for preparing and cleaning the log created by an object of class Trainer from the python library 'transformer's.

Usage

clean_pytorch_log_transformers(log)

Arguments

Value

Returns a data.frame containing epochs, loss, and val_loss.

See Also

Other Utils: auto_n_cores(), create_config_state(), create_dir(), generate_id(), get_file_extension(), get_py_package_versions(), is.null_or_na(), output_message(), print_message(), run_py_file()

Description

This function calculates different version of Cohen's Kappa.

Usage

cohens_kappa(rater_one, rater_two)

Arguments

Value

Returns a list containing the results for Cohen' Kappa if no weights are applied (kappa_unweighted), if weights are applied and the weights increase linear (kappa_linear), and if weights are applied and the weights increase quadratic (kappa_squared).

References

Cohen, J (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–220. doi:10.1037/h0026256

Cohen, J (1960). A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement, 20(1), 37–46. doi:10.1177/001316446002000104

See Also

Other performance measures: fleiss_kappa(), kendalls_w(), kripp_alpha()

Description

Check whether the passed dir_path directory exists. If not, creates a new directory and prints a msg message if trace is TRUE.

Usage

create_dir(dir_path, trace, msg = "Creating Directory", msg_fun = TRUE)

Arguments

Value

TRUE or FALSE depending on whether the shiny app is active.

See Also

Other Utils: auto_n_cores(), clean_pytorch_log_transformers(), create_config_state(), generate_id(), get_file_extension(), get_py_package_versions(), is.null_or_na(), output_message(), print_message(), run_py_file()

Description

Function for creating synthetic cases in order to balance the data for training with TEClassifierRegular or TEClassifierProtoNet]. This is an auxiliary function for use with get_synthetic_cases_from_matrix to allow parallel computations.

Usage

create_synthetic_units_from_matrix(
  matrix_form,
  target,
  required_cases,
  k,
  method,
  cat,
  k_s,
  max_k
)

Arguments

Value

Returns a list which contains the text embeddings of the new synthetic cases as a named data.frame and their labels as a named factor.

See Also

Other data_management_utils: get_n_chunks(), get_synthetic_cases_from_matrix()

Description

Abstract class for managing the data and samples during training a classifier. DataManagerClassifier is used with TEClassifierRegular and TEClassifierProtoNet.

Value

Objects of this class are used for ensuring the correct data management for training different types of classifiers. Objects of this class are also used for data augmentation by creating synthetic cases with different techniques.

Public fields

Methods

Public methods

• DataManagerClassifier$new()

• DataManagerClassifier$get_config()

• DataManagerClassifier$get_labeled_data()

• DataManagerClassifier$get_unlabeled_data()

• DataManagerClassifier$get_samples()

• DataManagerClassifier$set_state()

• DataManagerClassifier$get_n_folds()

• DataManagerClassifier$get_n_classes()

• DataManagerClassifier$get_statistics()

• DataManagerClassifier$get_dataset()

• DataManagerClassifier$get_val_dataset()

• DataManagerClassifier$get_test_dataset()

• DataManagerClassifier$create_synthetic()

• DataManagerClassifier$add_replace_pseudo_data()

• DataManagerClassifier$clone()

Method new()

Creating a new instance of this class.

Usage

DataManagerClassifier$new(
  data_embeddings,
  data_targets,
  folds = 5,
  val_size = 0.25,
  class_levels,
  one_hot_encoding = TRUE,
  add_matrix_map = TRUE,
  sc_methods = "dbsmote",
  sc_min_k = 1,
  sc_max_k = 10,
  trace = TRUE,
  n_cores = auto_n_cores()
)

Arguments

Returns

Method returns an initialized object of class DataManagerClassifier.

Method get_config()

Method for requesting the configuration of the DataManagerClassifier.

Usage

DataManagerClassifier$get_config()

Returns

Returns a list storing the configuration of the DataManagerClassifier.

Method get_labeled_data()

Method for requesting the complete labeled data set.

Usage

DataManagerClassifier$get_labeled_data()

Returns

Returns an object of class datasets.arrow_dataset.Dataset containing all cases with labels.

Method get_unlabeled_data()

Method for requesting the complete unlabeled data set.

Usage

DataManagerClassifier$get_unlabeled_data()

Returns

Returns an object of class datasets.arrow_dataset.Dataset containing all cases without labels.

Method get_samples()

Method for requesting the assignments to train, validation, and test data sets for every fold and the final training.

Usage

DataManagerClassifier$get_samples()

Returns

Returns a list storing the assignments to a train, validation, and test data set for every fold. In the case of the sample for the final training the test data set is always empty (NULL).

Method set_state()

Method for setting the current state of the DataManagerClassifier.

Usage

DataManagerClassifier$set_state(iteration, step = NULL)

Arguments

Returns

Method does not return anything. It is used for setting the internal state of the DataManager.

Method get_n_folds()

Method for requesting the number of folds the DataManagerClassifier can use with the current data.

Usage

DataManagerClassifier$get_n_folds()

Returns

Returns the number of folds the DataManagerClassifier uses.

Method get_n_classes()

Method for requesting the number of classes.

Usage

DataManagerClassifier$get_n_classes()

Returns

Returns the number classes.

Method get_statistics()

Method for requesting descriptive sample statistics.

Usage

DataManagerClassifier$get_statistics()

Returns

Returns a table describing the absolute frequencies of the labeled and unlabeled data. The rows contain the length of the sequences while the columns contain the labels.

Method get_dataset()

Method for requesting a data set for training depending in the current state of the DataManagerClassifier.

Usage

DataManagerClassifier$get_dataset(
  inc_labeled = TRUE,
  inc_unlabeled = FALSE,
  inc_synthetic = FALSE,
  inc_pseudo_data = FALSE
)

Arguments

Returns

Returns an object of class datasets.arrow_dataset.Dataset containing the requested kind of data along with all requested transformations for training. Please note that this method returns a data sets that is designed for training only. The corresponding validation data set is requested with get_val_dataset and the corresponding test data set with get_test_dataset.

Method get_val_dataset()

Method for requesting a data set for validation depending in the current state of the DataManagerClassifier.

Usage

DataManagerClassifier$get_val_dataset()

Returns

Returns an object of class datasets.arrow_dataset.Dataset containing the requested kind of data along with all requested transformations for validation. The corresponding data set for training can be requested with get_dataset and the corresponding data set for testing with get_test_dataset.

Method get_test_dataset()

Method for requesting a data set for testing depending in the current state of the DataManagerClassifier.

Usage

DataManagerClassifier$get_test_dataset()

Returns

Returns an object of class datasets.arrow_dataset.Dataset containing the requested kind of data along with all requested transformations for validation. The corresponding data set for training can be requested with get_dataset and the corresponding data set for validation with get_val_dataset.

Method create_synthetic()

Method for generating synthetic data used during training. The process uses all labeled data belonging to the current state of the DataManagerClassifier.

Usage

DataManagerClassifier$create_synthetic(trace = TRUE, inc_pseudo_data = FALSE)

Arguments

Returns

This method does nothing return. It generates a new data set for synthetic cases which are stored as an object of class datasets.arrow_dataset.Dataset in the field datasets$data_labeled_synthetic. Please note that a call of this method will override an existing data set in the corresponding field.

Method add_replace_pseudo_data()

Method for adding data with pseudo labels generated by a classifier

Usage

DataManagerClassifier$add_replace_pseudo_data(inputs, labels)

Arguments

Returns

This method does nothing return. It generates a new data set for synthetic cases which are stored as an object of class datasets.arrow_dataset.Dataset in the field datasets$data_labeled_pseudo. Please note that a call of this method will override an existing data set in the corresponding field.

Method clone()

The objects of this class are cloneable with this method.

Usage

DataManagerClassifier$clone(deep = FALSE)

Arguments

See Also

Other Data Management: EmbeddedText, LargeDataSetForText, LargeDataSetForTextEmbeddings

Description

Object of class R6 which stores the text embeddings generated by an object of class TextEmbeddingModel. The text embeddings are stored within memory/RAM. In the case of a high number of documents the data may not fit into memory/RAM. Thus, please use this object only for a small sample of texts. In general, it is recommended to use an object of class LargeDataSetForTextEmbeddings which can deal with any number of texts.

Value

Returns an object of class EmbeddedText. These objects are used for storing and managing the text embeddings created with objects of class TextEmbeddingModel. Objects of class EmbeddedText serve as input for objects of class TEClassifierRegular, TEClassifierProtoNet, and TEFeatureExtractor. The main aim of this class is to provide a structured link between embedding models and classifiers. Since objects of this class save information on the text embedding model that created the text embedding it ensures that only embedding generated with same embedding model are combined. Furthermore, the stored information allows objects to check if embeddings of the correct text embedding model are used for training and predicting.

Public fields

Methods

Public methods

• EmbeddedText$configure()

• EmbeddedText$save()

• EmbeddedText$is_configured()

• EmbeddedText$load_from_disk()

• EmbeddedText$get_model_info()

• EmbeddedText$get_model_label()

• EmbeddedText$get_times()

• EmbeddedText$get_features()

• EmbeddedText$get_original_features()

• EmbeddedText$is_compressed()

• EmbeddedText$add_feature_extractor_info()

• EmbeddedText$get_feature_extractor_info()

• EmbeddedText$convert_to_LargeDataSetForTextEmbeddings()

• EmbeddedText$n_rows()

• EmbeddedText$get_all_fields()

• EmbeddedText$clone()

Method configure()

Creates a new object representing text embeddings.

Usage

EmbeddedText$configure(
  model_name = NA,
  model_label = NA,
  model_date = NA,
  model_method = NA,
  model_version = NA,
  model_language = NA,
  param_seq_length = NA,
  param_chunks = NULL,
  param_features = NULL,
  param_overlap = NULL,
  param_emb_layer_min = NULL,
  param_emb_layer_max = NULL,
  param_emb_pool_type = NULL,
  param_aggregation = NULL,
  embeddings
)

Arguments

Returns

Returns an object of class EmbeddedText which stores the text embeddings produced by an objects of class TextEmbeddingModel.

Method save()

Saves a data set to disk.

Usage

EmbeddedText$save(dir_path, folder_name, create_dir = TRUE)

Arguments

Returns

Method does not return anything. It write the data set to disk.

Method is_configured()

Method for checking if the model was successfully configured. An object can only be used if this value is TRUE.

Usage

EmbeddedText$is_configured()

Returns

bool TRUE if the model is fully configured. FALSE if not.

Method load_from_disk()

loads an object of class EmbeddedText from disk and updates the object to the current version of the package.

Usage

EmbeddedText$load_from_disk(dir_path)

Arguments

Returns

Method does not return anything. It loads an object from disk.

Method get_model_info()

Method for retrieving information about the model that generated this embedding.

Usage

EmbeddedText$get_model_info()

Returns

list contains all saved information about the underlying text embedding model.

Method get_model_label()

Method for retrieving the label of the model that generated this embedding.

Usage

EmbeddedText$get_model_label()

Returns

string Label of the corresponding text embedding model

Method get_times()

Number of chunks/times of the text embeddings.

Usage

EmbeddedText$get_times()

Returns

Returns an int describing the number of chunks/times of the text embeddings.

Method get_features()

Number of actual features/dimensions of the text embeddings.In the case a feature extractor was used the number of features is smaller as the original number of features. To receive the original number of features (the number of features before applying a feature extractor) you can use the method get_original_features of this class.

Usage

EmbeddedText$get_features()

Returns

Returns an int describing the number of features/dimensions of the text embeddings.

Method get_original_features()

Number of original features/dimensions of the text embeddings.

Usage

EmbeddedText$get_original_features()

Returns

Returns an int describing the number of features/dimensions if no feature extractor) is used or before a feature extractor) is applied.

Method is_compressed()

Checks if the text embedding were reduced by a feature extractor.

Usage

EmbeddedText$is_compressed()

Returns

Returns TRUE if the number of dimensions was reduced by a feature extractor. If not return FALSE.

Method add_feature_extractor_info()

Method setting information on the feature extractor that was used to reduce the number of dimensions of the text embeddings. This information should only be used if a feature extractor was applied.

Usage

EmbeddedText$add_feature_extractor_info(
  model_name,
  model_label = NA,
  features = NA,
  method = NA,
  noise_factor = NA,
  optimizer = NA
)

Arguments

Returns

Method does nothing return. It sets information on a feature extractor.

Method get_feature_extractor_info()

Method for receiving information on the feature extractor that was used to reduce the number of dimensions of the text embeddings.

Usage

EmbeddedText$get_feature_extractor_info()

Returns

Returns a list with information on the feature extractor. If no feature extractor was used it returns NULL.

Method convert_to_LargeDataSetForTextEmbeddings()

Method for converting this object to an object of class LargeDataSetForTextEmbeddings.

Usage

EmbeddedText$convert_to_LargeDataSetForTextEmbeddings()

Returns

Returns an object of class LargeDataSetForTextEmbeddings which uses memory mapping allowing to work with large data sets.

Method n_rows()

Number of rows.

Usage

EmbeddedText$n_rows()

Returns

Returns the number of rows of the text embeddings which represent the number of cases.

Method get_all_fields()

Return all fields.

Usage

EmbeddedText$get_all_fields()

Returns

Method returns a list containing all public and private fields of the object.

Method clone()

The objects of this class are cloneable with this method.

Usage

EmbeddedText$clone(deep = FALSE)

Arguments

See Also

Other Data Management: DataManagerClassifier, LargeDataSetForText, LargeDataSetForTextEmbeddings

Description

This function calculates Fleiss' Kappa.

Usage

fleiss_kappa(rater_one, rater_two, additional_raters = NULL)

Arguments

Value

Retuns the value for Fleiss' Kappa.

References

Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378–382. doi:10.1037/h0031619

See Also

Other performance measures: cohens_kappa(), kendalls_w(), kripp_alpha()

Description

Function for generating an ID suffix for objects of class TextEmbeddingModel, TEClassifierRegular, and TEClassifierProtoNet.

Usage

generate_id(length = 16)

Arguments

Value

Returns a string of the requested length.

See Also

Other Utils: auto_n_cores(), clean_pytorch_log_transformers(), create_config_state(), create_dir(), get_file_extension(), get_py_package_versions(), is.null_or_na(), output_message(), print_message(), run_py_file()

Description

Function for requesting a vector containing the alpha-3 codes for most countries.

Usage

get_alpha_3_codes()

Value

Returns a vector containing the alpha-3 codes for most countries.

See Also

Other Auxiliary Functions: matrix_to_array_c(), summarize_tracked_sustainability(), to_categorical_c()

Description

This function calculates different reliability measures which are based on the empirical research method of content analysis.

Usage

get_coder_metrics(
  true_values = NULL,
  predicted_values = NULL,
  return_names_only = FALSE
)

Arguments

Value

If return_names_only = FALSE returns a vector with the following reliability measures:

• iota_index: Iota Index from the Iota Reliability Concept Version 2.

• min_iota2: Minimal Iota from Iota Reliability Concept Version 2.

• avg_iota2: Average Iota from Iota Reliability Concept Version 2.

• max_iota2: Maximum Iota from Iota Reliability Concept Version 2.

• min_alpha: Minmal Alpha Reliability from Iota Reliability Concept Version 2.

• avg_alpha: Average Alpha Reliability from Iota Reliability Concept Version 2.

• max_alpha: Maximum Alpha Reliability from Iota Reliability Concept Version 2.

• static_iota_index: Static Iota Index from Iota Reliability Concept Version 2.

• dynamic_iota_index: Dynamic Iota Index Iota Reliability Concept Version 2.

• kalpha_nominal: Krippendorff's Alpha for nominal variables.

• kalpha_ordinal: Krippendorff's Alpha for ordinal variables.

• kendall: Kendall's coefficient of concordance W with correction for ties.

• c_kappa_unweighted: Cohen's Kappa unweighted.

• c_kappa_linear: Weighted Cohen's Kappa with linear increasing weights.

• c_kappa_squared: Weighted Cohen's Kappa with quadratic increasing weights.

• kappa_fleiss: Fleiss' Kappa for multiple raters without exact estimation.

• percentage_agreement: Percentage Agreement.

• balanced_accuracy: Average accuracy within each class.

• gwet_ac: Gwet's AC1/AC2 agreement coefficient.

If return_names_only = TRUE returns only the names of the vector elements.

See Also

Other classifier_utils: calc_standard_classification_measures()

Description

Function for requesting the file extension

Usage

get_file_extension(file_path)

Arguments

Value

Returns the extension of a file as a string.

See Also

Other Utils: auto_n_cores(), clean_pytorch_log_transformers(), create_config_state(), create_dir(), generate_id(), get_py_package_versions(), is.null_or_na(), output_message(), print_message(), run_py_file()

Description

Function for calculating the number of chunks/sequences for every case.

Usage

get_n_chunks(text_embeddings, features, times)

Arguments

Value

Namedvector of integers representing the number of chunks/sequences for every case.

See Also

Other data_management_utils: create_synthetic_units_from_matrix(), get_synthetic_cases_from_matrix()

Description

Function for requesting a summary of the versions of all critical python components.

Usage

get_py_package_versions()

Value

Returns a list that contains the version number of python and the versions of critical python packages. If a package is not available version is set to NA.

See Also

Other Utils: auto_n_cores(), clean_pytorch_log_transformers(), create_config_state(), create_dir(), generate_id(), get_file_extension(), is.null_or_na(), output_message(), print_message(), run_py_file()

Description

This function creates synthetic cases for balancing the training with an object of the class TEClassifierRegular or TEClassifierProtoNet.

Usage

get_synthetic_cases_from_matrix(
  matrix_form,
  times,
  features,
  target,
  sequence_length,
  method = c("smote"),
  min_k = 1,
  max_k = 6
)

Arguments

Value

list with the following components:

• syntetic_embeddings: Named data.frame containing the text embeddings of the synthetic cases.

• syntetic_targets: Named factor containing the labels of the corresponding synthetic cases.

• n_syntetic_units: table showing the number of synthetic cases for every label/category.

See Also

Other data_management_utils: create_synthetic_units_from_matrix(), get_n_chunks()

Description

Function for installing 'aifeducation' on a machine. This functions assumes that not 'python' and no 'miniconda' is installed. Only'pytorch' is installed.

Usage

install_aifeducation(install_aifeducation_studio = TRUE)

Arguments

Value

Function does nothing return. It installs python, optional R packages, and necessary 'python' packages on a machine.

See Also

Other Installation and Configuration: check_aif_py_modules(), install_py_modules(), set_transformers_logger()

Description

Function for installing the necessary python modules.

Usage

install_py_modules(
  envname = "aifeducation",
  install = "pytorch",
  transformer_version = "<=4.46",
  tokenizers_version = "<=0.20.4",
  pandas_version = "<=2.2.3",
  datasets_version = "<=3.1.0",
  codecarbon_version = "<=2.8.2",
  safetensors_version = "<=0.4.5",
  torcheval_version = "<=0.0.7",
  accelerate_version = "<=1.1.1",
  pytorch_cuda_version = "12.1",
  python_version = "3.9",
  remove_first = FALSE
)

Arguments

Value

Returns no values or objects. Function is used for installing the necessary python libraries in a conda environment.

See Also

Other Installation and Configuration: check_aif_py_modules(), install_aifeducation(), set_transformers_logger()

Description

Function for checking if an object is NULL or .

Usage

is.null_or_na(object)

Arguments

Value

Returns FALSE if the object is not NULL and not NA. Returns TRUE in all other cases.

See Also

Other Utils: auto_n_cores(), clean_pytorch_log_transformers(), create_config_state(), create_dir(), generate_id(), get_file_extension(), get_py_package_versions(), output_message(), print_message(), run_py_file()

Description

This function calculates Kendall's coefficient of concordance w with and without correction.

Usage

kendalls_w(rater_one, rater_two, additional_raters = NULL)

Arguments

Value

Returns a list containing the results for Kendall's coefficient of concordance w with and without correction.

See Also

Other performance measures: cohens_kappa(), fleiss_kappa(), kripp_alpha()

Description

This function calculates different Krippendorff's Alpha for nominal and ordinal variables.

Usage

kripp_alpha(rater_one, rater_two, additional_raters = NULL)

Arguments

Value

Returns a list containing the results for Krippendorff's Alpha for nominal and ordinal data.

References

Krippendorff, K. (2019). Content Analysis: An Introduction to Its Methodology (4th Ed.). SAGE

See Also

Other performance measures: cohens_kappa(), fleiss_kappa(), kendalls_w()

Description

This object contains public and private methods which may be useful for every large data sets. Objects of this class are not intended to be used directly. LargeDataSetForTextEmbeddings or LargeDataSetForText.

Value

Returns a new object of this class.

Methods

Public methods

• LargeDataSetBase$n_cols()

• LargeDataSetBase$n_rows()

• LargeDataSetBase$get_colnames()

• LargeDataSetBase$get_dataset()

• LargeDataSetBase$reduce_to_unique_ids()

• LargeDataSetBase$select()

• LargeDataSetBase$get_ids()

• LargeDataSetBase$save()

• LargeDataSetBase$load_from_disk()

• LargeDataSetBase$load()

• LargeDataSetBase$get_all_fields()

• LargeDataSetBase$clone()

Method n_cols()

Number of columns in the data set.

Usage

LargeDataSetBase$n_cols()

Returns

int describing the number of columns in the data set.

Method n_rows()

Number of rows in the data set.

Usage

LargeDataSetBase$n_rows()

Returns

int describing the number of rows in the data set.

Method get_colnames()

Get names of the columns in the data set.

Usage

LargeDataSetBase$get_colnames()

Returns

vector containing the names of the columns as strings.

Method get_dataset()

Get data set.

Usage

LargeDataSetBase$get_dataset()

Returns

Returns the data set of this object as an object of class datasets.arrow_dataset.Dataset.

Method reduce_to_unique_ids()

Reduces the data set to a data set containing only unique ids. In the case an id exists multiple times in the data set the first case remains in the data set. The other cases are dropped.

Attention Calling this method will change the data set in place.

Usage

LargeDataSetBase$reduce_to_unique_ids()

Returns

Method does not return anything. It changes the data set of this object in place.

Method select()

Returns a data set which contains only the cases belonging to the specific indices.

Usage

LargeDataSetBase$select(indicies)

Arguments

Returns

Returns a data set of class datasets.arrow_dataset.Dataset with the selected rows.

Method get_ids()

Get ids

Usage

LargeDataSetBase$get_ids()

Returns

Returns a vector containing the ids of every row as strings.

Method save()

Saves a data set to disk.

Usage

LargeDataSetBase$save(dir_path, folder_name, create_dir = TRUE)

Arguments

Returns

Method does not return anything. It write the data set to disk.

Method load_from_disk()

loads an object of class LargeDataSetBase from disk 'and updates the object to the current version of the package.

Usage

LargeDataSetBase$load_from_disk(dir_path)

Arguments

Returns

Method does not return anything. It loads an object from disk.

Method load()

Loads a data set from disk.

Usage

LargeDataSetBase$load(dir_path)

Arguments

Returns

Method does not return anything. It loads a data set from disk.

Method get_all_fields()

Return all fields.

Usage

LargeDataSetBase$get_all_fields()

Returns

Method returns a list containing all public and private fields of the object.

Method clone()

The objects of this class are cloneable with this method.

Usage

LargeDataSetBase$clone(deep = FALSE)

Arguments

Description

This object stores raw texts. The data of this objects is not stored in memory directly. By using memory mapping these objects allow to work with data sets which do not fit into memory/RAM.

Value

Returns a new object of this class.

Super class

aifeducation::LargeDataSetBase -> LargeDataSetForText

Methods

Public methods

• LargeDataSetForText$new()

• LargeDataSetForText$add_from_files_txt()

• LargeDataSetForText$add_from_files_pdf()

• LargeDataSetForText$add_from_files_xlsx()

• LargeDataSetForText$add_from_data.frame()

• LargeDataSetForText$get_private()

• LargeDataSetForText$clone()

Method new()

Method for creation of LargeDataSetForText instance. It can be initialized with init_data parameter if passed (Uses add_from_data.frame() method if init_data is data.frame).

Usage

LargeDataSetForText$new(init_data = NULL)

Arguments

Returns

A new instance of this class initialized with init_data if passed.

Method add_from_files_txt()

Method for adding raw texts saved within .txt files to the data set. Please note the the directory should contain one folder for each .txt file. In order to create an informative data set every folder can contain the following additional files:

• bib_entry.txt: containing a text version of the bibliographic information of the raw text.

• license.txt: containing a statement about the license to use the raw text such as "CC BY".

• url_license.txt: containing the url/link to the license in the internet.

• text_license.txt: containing the license in raw text.

• url_source.txt: containing the url/link to the source in the internet.

  The id of every .txt file is the file name without file extension. Please be aware to provide unique file names. Id and raw texts are mandatory, bibliographic and license information are optional.

Usage

LargeDataSetForText$add_from_files_txt(
  dir_path,
  batch_size = 500,
  log_file = NULL,
  log_write_interval = 2,
  log_top_value = 0,
  log_top_total = 1,
  log_top_message = NA,
  trace = TRUE
)

Arguments

Returns

The method does not return anything. It adds new raw texts to the data set.

Method add_from_files_pdf()

Method for adding raw texts saved within .pdf files to the data set. Please note the the directory should contain one folder for each .pdf file. In order to create an informative data set every folder can contain the following additional files:

• bib_entry.txt: containing a text version of the bibliographic information of the raw text.

• license.txt: containing a statement about the license to use the raw text such as "CC BY".

• url_license.txt: containing the url/link to the license in the internet.

• text_license.txt: containing the license in raw text.

• url_source.txt: containing the url/link to the source in the internet.

  The id of every .pdf file is the file name without file extension. Please be aware to provide unique file names. Id and raw texts are mandatory, bibliographic and license information are optional.

Usage

LargeDataSetForText$add_from_files_pdf(
  dir_path,
  batch_size = 500,
  log_file = NULL,
  log_write_interval = 2,
  log_top_value = 0,
  log_top_total = 1,
  log_top_message = NA,
  trace = TRUE
)

Arguments

Returns

The method does not return anything. It adds new raw texts to the data set.

Method add_from_files_xlsx()

Method for adding raw texts saved within .xlsx files to the data set. The method assumes that the texts are saved in the rows and that the columns store the id and the raw texts in the columns. In addition, a column for the bibliography information and the license can be added. The column names for these rows must be specified with the following arguments. They must be the same for all .xlsx files in the chosen directory. Id and raw texts are mandatory, bibliographic, license, license's url, license's text, and source's url are optional. Additional columns are dropped.

Usage

LargeDataSetForText$add_from_files_xlsx(
  dir_path,
  trace = TRUE,
  id_column = "id",
  text_column = "text",
  bib_entry_column = "bib_entry",
  license_column = "license",
  url_license_column = "url_license",
  text_license_column = "text_license",
  url_source_column = "url_source",
  log_file = NULL,
  log_write_interval = 2,
  log_top_value = 0,
  log_top_total = 1,
  log_top_message = NA
)

Arguments

Returns

The method does not return anything. It adds new raw texts to the data set.

Method add_from_data.frame()

Method for adding raw texts from a data.frame

Usage

LargeDataSetForText$add_from_data.frame(data_frame)

Arguments

Returns

The method does not return anything. It adds new raw texts to the data set.

Method get_private()

Method for requesting all private fields and methods. Used for loading and updating an object.

Usage

LargeDataSetForText$get_private()

Returns

Returns a list with all private fields and methods.

Method clone()

The objects of this class are cloneable with this method.

Usage

LargeDataSetForText$clone(deep = FALSE)

Arguments

See Also

Other Data Management: DataManagerClassifier, EmbeddedText, LargeDataSetForTextEmbeddings

Description

This object stores text embeddings which are usually produced by an object of class TextEmbeddingModel. The data of this objects is not stored in memory directly. By using memory mapping these objects allow to work with data sets which do not fit into memory/RAM.

LargeDataSetForTextEmbeddings are used for storing and managing the text embeddings created with objects of class TextEmbeddingModel. Objects of class LargeDataSetForTextEmbeddings serve as input for objects of class TEClassifierRegular, TEClassifierProtoNet, and TEFeatureExtractor. The main aim of this class is to provide a structured link between embedding models and classifiers. Since objects of this class save information on the text embedding model that created the text embedding it ensures that only embedding generated with same embedding model are combined. Furthermore, the stored information allows objects to check if embeddings of the correct text embedding model are used for training and predicting.

Value

Returns a new object of this class.

Super class

aifeducation::LargeDataSetBase -> LargeDataSetForTextEmbeddings

Methods

Public methods

• LargeDataSetForTextEmbeddings$configure()

• LargeDataSetForTextEmbeddings$is_configured()

• LargeDataSetForTextEmbeddings$get_text_embedding_model_name()

• LargeDataSetForTextEmbeddings$get_model_info()

• LargeDataSetForTextEmbeddings$load_from_disk()

• LargeDataSetForTextEmbeddings$get_model_label()

• LargeDataSetForTextEmbeddings$add_feature_extractor_info()

• LargeDataSetForTextEmbeddings$get_feature_extractor_info()

• LargeDataSetForTextEmbeddings$is_compressed()

• LargeDataSetForTextEmbeddings$get_times()

• LargeDataSetForTextEmbeddings$get_features()

• LargeDataSetForTextEmbeddings$get_original_features()

• LargeDataSetForTextEmbeddings$add_embeddings_from_array()

• LargeDataSetForTextEmbeddings$add_embeddings_from_EmbeddedText()

• LargeDataSetForTextEmbeddings$add_embeddings_from_LargeDataSetForTextEmbeddings()

• LargeDataSetForTextEmbeddings$convert_to_EmbeddedText()

• LargeDataSetForTextEmbeddings$clone()

Method configure()

Creates a new object representing text embeddings.

Usage

LargeDataSetForTextEmbeddings$configure(
  model_name = NA,
  model_label = NA,
  model_date = NA,
  model_method = NA,
  model_version = NA,
  model_language = NA,
  param_seq_length = NA,
  param_chunks = NULL,
  param_features = NULL,
  param_overlap = NULL,
  param_emb_layer_min = NULL,
  param_emb_layer_max = NULL,
  param_emb_pool_type = NULL,
  param_aggregation = NULL
)

Arguments

Returns

The method returns a new object of this class.

Method is_configured()

Method for checking if the model was successfully configured. An object can only be used if this value is TRUE.

Usage

LargeDataSetForTextEmbeddings$is_configured()

Returns

bool TRUE if the model is fully configured. FALSE if not.

Method get_text_embedding_model_name()

Method for requesting the name (unique id) of the underlying text embedding model.

Usage

LargeDataSetForTextEmbeddings$get_text_embedding_model_name()

Returns

Returns a string describing name of the text embedding model.

Method get_model_info()

Method for retrieving information about the model that generated this embedding.

Usage

LargeDataSetForTextEmbeddings$get_model_info()

Returns

list containing all saved information about the underlying text embedding model.

Method load_from_disk()

loads an object of class LargeDataSetForTextEmbeddings from disk and updates the object to the current version of the package.

Usage

LargeDataSetForTextEmbeddings$load_from_disk(dir_path)

Arguments

Returns

Method does not return anything. It loads an object from disk.

Method get_model_label()

Method for retrieving the label of the model that generated this embedding.

Usage

LargeDataSetForTextEmbeddings$get_model_label()