Functions
	model_statistics (J_history, predicted, ground_truth, pred_vector, gt_vector, f1_list, recall_list, precision_list, accuracy_list, train_or_val="train")
	Evaluates model with metrics, such as accuracy and f1_score.

	right_size (CUs)

	compute_conf_matrix (predicted, ground_truth)
	Computes the confusion matrix.

	compute_top_k_accuracy (pred_vector, gt_vector, topk)
	Computes the top k accuracy score.

	compute_num_splits_sent (pred_lst)
	Computes the num of splits that would be analyzed by the encoder.

	compute_multi_thres_performance (pred_lst, gt_lst)
	Computes multi-threshold performance.

	compute_ROC_curve (pred_vector, gt_vector, pred_num)
	Computes ROC curve.

	model_simple_metrics (predicted, ground_truth)
	Evaluates model with metrics 4 metrics, such as accuracy, f1_score, recall and precision.

	obtain_best_modes (rs, pred)
	Converts a prediction into a specific number that corresponds to the best way to split (non-split, quad tree, binary vert tree...)

	obtain_mode (pred)
	Converts a prediction into a specific number that corresponds to the best way to split (non-split, quad tree, binary vert tree...)

	one_hot_enc (tensor, num_classes=6)
	Implements one-hot encoding to a specific tensor with the set of split modes.

	print_parameters (model, optimizer)
	Prints the parameters from the state dictionaries of the model and optimizer.

	save_model_parameters (dir_name, f_name, model)
	Saves only the model parameters to a specific folder.

	save_model (dir_name, f_name, model, optimizer, loss, acc)
	Saves the parameters of the model and of the optimizer, and also the loss and the accuracy.

	load_model_parameters_stg (model, path, stg, dev)
	Loads all stages but make sure that the stage number 'stg' has the same parameters has the previous.

	load_model_parameters_eval (model, path, dev)
	Loads all stages, meant to be used with the eval_model script.

	load_model_stg_12_stg_3 (model, path, dev)
	THis function makes it possible to load parameters from the first and second stage to the third.

	load_model_stg_3_stg_4 (model, path, dev)
	This function makes it possible to load parameters from the third stage to the fourth.

	load_model_stg_4_stg_5 (model, path, dev)
	This function makes it possible to load parameters from the fourth stage to the fith.

	load_model_stg_5_stg_6 (model, path, dev)
	This function makes it possible to load parameters from the fourth stage to the fith.

	print_current_time ()
	Prints current time.

Detailed Description

@package docstring 

@file train_model_utils.py 

@brief Group of functions that are supposed to be used directly in the training or evaluation scripts
 
@section libraries_train_model_utils Libraries 
- os
- torch
- numpy
- matplotlib.pyplot
- dataset_utils
- seaborn
- itertools
- datetime
- sklearn.metrics

@section classes_train_model_utils Classes 
- None
 
@section functions_train_model_utils Functions 
- model_statistics(J_history, predicted, ground_truth, pred_vector, gt_vector,
- compute_conf_matrix(predicted, ground_truth)
- compute_top_k_accuracy(pred_vector, gt_vector, topk)
- compute_num_splits_sent(pred_lst)
- compute_multi_thres_performance(pred_lst, gt_lst)
- compute_ROC_curve(pred_vector, gt_vector, pred_num)
- model_simple_metrics(predicted, ground_truth)
- obtain_best_modes(rs, pred)
- obtain_mode(pred)
- one_hot_enc(tensor, num_classes=6)
- print_parameters(model, optimizer)
- save_model_parameters(dir_name, f_name, model)
- save_model(dir_name, f_name, model, optimizer, loss, acc)
- load_model_parameters_stg(model, path, stg, dev)
- load_model_parameters_eval(model, path, dev)
- load_model_stg_12_stg_3(model, path, dev)
- load_model_stg_3_stg_4(model, path, dev)
- load_model_stg_4_stg_5(model, path, dev)
- load_model_stg_5_stg_6(model, path, dev)
- print_current_time()
 
@section global_vars_train_model_utils Global Variables 
- None

@section todo_train_model_utils TODO 
- None
    
@section license License 
MIT License 
Copyright (c) 2022 Raul Kevin do Espirito Santo Viana
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

@section author_train_model_utils Author(s)
- Created by Raul Kevin Viana
- Last time modified is 2023-01-29 22:22:04.154941

Function Documentation

◆ compute_conf_matrix()

msecnn_raulkviana.train_model_utils.compute_conf_matrix	(	predicted,
		ground_truth
	)

Computes the confusion matrix.

Parameters

[in]	predicted	List of predictions made by the model with single value
[in]	ground_truth	List of the ground-truths with single value
[out]	accuracy	Accuracy score

◆ compute_multi_thres_performance()

msecnn_raulkviana.train_model_utils.compute_multi_thres_performance	(	pred_lst,
		gt_lst
	)

Computes multi-threshold performance.

Parameters

[in]	predicted	List of predictions made by the model with integer value
[in]	ground_truth	List of the ground-truths with single value
[out]	res	Accuracy score

◆ compute_num_splits_sent()

msecnn_raulkviana.train_model_utils.compute_num_splits_sent ( pred_lst )

Computes the num of splits that would be analyzed by the encoder.

Parameters

[in]	predicted	List of predictions made by the model with probabilities values
[out]	res	Mean of number of splits sent

◆ compute_ROC_curve()

msecnn_raulkviana.train_model_utils.compute_ROC_curve	(	pred_vector,
		gt_vector,
		pred_num
	)

Computes ROC curve.

Parameters

[in]	pred_vector	List of predictions vectors (one-hot encoded)
[in]	gt_vector	List of the ground-truths vectors (one-hot encoded)
[in]	pred_num	List of the predicitons with numbers corresponding to partitions

Returns: [out] figure: Figure with the ROC curve

◆ compute_top_k_accuracy()

msecnn_raulkviana.train_model_utils.compute_top_k_accuracy	(	pred_vector,
		gt_vector,
		topk
	)

Computes the top k accuracy score.

Parameters

[in]	predicted	List of predictions made by the model with probabilities for each split (pytorch tensor)
[in]	ground_truth	List of the ground-truths with single value (pytorch tensor)
[in]	topk	Number of best accuricies to choose
[out]	accuracy	Accuracy score

◆ load_model_parameters_eval()

msecnn_raulkviana.train_model_utils.load_model_parameters_eval	(	model,
		path,
		dev
	)

Loads all stages, meant to be used with the eval_model script.

Parameters

[in]	model	Model which the parameters will be loaded
[in]	path	Path/Folder containing the files that are supposed to be loaded
[in]	dev	Device do load the model to
[out]	model	Model loaded tuple

◆ load_model_parameters_stg()

msecnn_raulkviana.train_model_utils.load_model_parameters_stg	(	model,
		path,
		stg,
		dev
	)

Loads all stages but make sure that the stage number 'stg' has the same parameters has the previous.

Parameters

[in]	model	Model which the parameters will be loaded
[in]	path	Path/Folder containing the files that are supposed to be loaded
[in]	stg	Integer containing the last stage number to load
[in]	dev	Device do load the model to
[out]	model	Model loaded tuple

◆ load_model_stg_12_stg_3()

msecnn_raulkviana.train_model_utils.load_model_stg_12_stg_3	(	model,
		path,
		dev
	)

THis function makes it possible to load parameters from the first and second stage to the third.

Parameters

[in]	model	Model which the parameters will be loaded, with 2 models (one for the first and second stage, and another for the third stage)
[in]	path	Path/Folder containing the files that are supposed to be loaded
[in]	dev	Device to load the model to

◆ load_model_stg_3_stg_4()

msecnn_raulkviana.train_model_utils.load_model_stg_3_stg_4	(	model,
		path,
		dev
	)

This function makes it possible to load parameters from the third stage to the fourth.

Parameters

[in]	model	Model which the parameters will be loaded, with 2 models (one for the first and second stage, and another for the third stage)
[in]	path	Path/Folder containing the files that are supposed to be loaded
[in]	dev	Device to load the model to

◆ load_model_stg_4_stg_5()

msecnn_raulkviana.train_model_utils.load_model_stg_4_stg_5	(	model,
		path,
		dev
	)

This function makes it possible to load parameters from the fourth stage to the fith.

Parameters

[in]	model	Model which the parameters will be loaded, with 2 models (one for the first and second stage, and another for the third stage)
[in]	path	Path/Folder containing the files that are supposed to be loaded
[in]	dev	Device to load the model to

◆ load_model_stg_5_stg_6()

msecnn_raulkviana.train_model_utils.load_model_stg_5_stg_6	(	model,
		path,
		dev
	)

This function makes it possible to load parameters from the fourth stage to the fith.

Parameters

[in]	model	Model which the parameters will be loaded, with 2 models (one for the first and second stage, and another for the third stage)
[in]	path	Path/Folder containing the files that are supposed to be loaded
[in]	dev	Device to load the model to

◆ model_simple_metrics()

msecnn_raulkviana.train_model_utils.model_simple_metrics	(	predicted,
		ground_truth
	)

Evaluates model with metrics 4 metrics, such as accuracy, f1_score, recall and precision.

Parameters

[in]	predicted	List of predictions made by the model with single value
[in]	ground_truth	List of the ground-truths with single value
[out]	f1	F1 score
[out]	recall	Recall score
[out]	precision	Precision score
[out]	accuracy	Accuracy score

◆ model_statistics()

msecnn_raulkviana.train_model_utils.model_statistics	(	J_history,
		predicted,
		ground_truth,
		pred_vector,
		gt_vector,
		f1_list,
		recall_list,
		precision_list,
		accuracy_list,
		train_or_val = `"train"`
	)

Evaluates model with metrics, such as accuracy and f1_score.

This version plots the evolution of the metrics: f1-score, recall, precision, accuracy.

Parameters

[in]	J_history	Loss function values over iterations
[in]	predicted	List of predictions made by the model with single value
[in]	ground_truth	List of the ground-truths with single value
[in]	pred_vector	List of predictions made by the model with vectors values
[in]	gt_vector	List of the ground-truths with vectors values
[in]	train_or_val	String that is used to write on the image files names
[out]	f1	F1 score
[out]	recall	Recall score
[out]	precision	Precision score
[out]	accuracy	Accuracy score

◆ obtain_best_modes()

msecnn_raulkviana.train_model_utils.obtain_best_modes	(	rs,
		pred
	)

Converts a prediction into a specific number that corresponds to the best way to split (non-split, quad tree, binary vert tree...)

Parameters

[in]	rs	Thresholds
[in]	pred	Predicted values from the model with one-hot encoding
[out]	mode	Predicted values with the number of the mode

◆ obtain_mode()

msecnn_raulkviana.train_model_utils.obtain_mode ( pred )

Converts a prediction into a specific number that corresponds to the best way to split (non-split, quad tree, binary vert tree...)

Parameters

[in]	pred	Predicted values from the model with one-hot encoding
[out]	mode	Predicted values with the number of the mode

◆ one_hot_enc()

msecnn_raulkviana.train_model_utils.one_hot_enc	(	tensor,
		num_classes = `6`
	)

Implements one-hot encoding to a specific tensor with the set of split modes.

Parameters

[in]	tensor	Tensor with a set of split modes
[in]	num_classes	Number classes in the tensor
[out]	new_tensor	Tensor with one-hot encoding implemented

◆ print_current_time()

msecnn_raulkviana.train_model_utils.print_current_time ( )

Prints current time.

◆ print_parameters()

msecnn_raulkviana.train_model_utils.print_parameters	(	model,
		optimizer
	)

Prints the parameters from the state dictionaries of the model and optimizer.

Parameters

[in]	model	Model that the parameters will be printed
[in]	optimizer	Optimizer that the parameters will be printed

◆ right_size()

msecnn_raulkviana.train_model_utils.right_size ( CUs )

@brief Verify if the CU as the right size: height as to be lower than width

@param [in] CUs: Feature maps
@param [out] Boolean value indicating the right size

◆ save_model()

msecnn_raulkviana.train_model_utils.save_model	(	dir_name,
		f_name,
		model,
		optimizer,
		loss,
		acc
	)

Saves the parameters of the model and of the optimizer, and also the loss and the accuracy.

These are saved into the folder specified by the user.

Parameters

[in]	dir_name	Name of the directory where the parameters will be saved
[in]	f_name	Name of the file that the parameters will be saved on
[in]	model	Model which the parameters will be saved
[in]	optimizer	Optimizer which the parameters will be saved
[in]	loss	Loss value
[in]	acc	Accuracy value

◆ save_model_parameters()

msecnn_raulkviana.train_model_utils.save_model_parameters	(	dir_name,
		f_name,
		model
	)

Saves only the model parameters to a specific folder.

Parameters

[in]	dir_name	Name of the directory where the parameters will be saved
[in]	f_name	Name of the file that the parameters will be saved on
[in]	model	Model which the parameters will be saved