Deep Neural Network - Training and Testing Performance Comparison

Sep 3, 2026

A performance comparison study of deep neural network architectures for source-to-target task transfer across varying dataset sizes. Experiments define source task (AV) and target task (S4), using train/test splits and structured embeddings. Results report multiple accuracy metrics (word accuracy, template word accuracy, F-score, and line accuracy) alongside processing time for training and testing. Configurations evaluate DTNN against hybrid CRF-DNN-DTNN variants to assess scalability and effectiveness under different numbers of logs/templates/clusters.

LanguageEnglish

CategoryPresentations

FormatPDF

Pages6

Reading Time2 min

AccessFree to download

Experimental Setup

Source and Target Tasks

Models and Embedding Network

Dataset Partitions and Scales

Results and Metrics

Accuracy Metrics

Processing Time (Training/Testing)

| yyyy | on | | pts/0 | | --- | --- | --- | --- | | | Deep Neural Network (DNN) | | | --- | --- | --- | | Embedding

Neural Network
Char Network
| | | | | Source Task (AV) | | Target Task (S4) | | | --- | --- | --- | --- | --- | | | Train (SX) | Test (SY) | Train (TX) | Test (TY) | | \# Logs | 100000 | 100000 | 74496 | 74910 | | \# Templates | 100000 | 100000 | 74496 | 74910 | | \# Clusters | 38 | 45 | 41 | 75 | | \# Train.
Data | Methods | Accuracy Metrics | | | | Processing time (sec) | | | --- | --- | --- | --- | --- | --- | --- | --- | | | | Word Acc. | Template Word Acc. | F Acc. | Line Acc. | Training T. | Testing T. | | 0 | DTNN | 0.898 􀀆 8.73E-7 | 0.838 􀀆 5.45E-6 | 0.797 􀀆 1.07E-5 | 0.341 􀀆 7.04E-6 | 0.341 􀀆 7.04E-6 | 0.341 􀀆 7.04E-6 | | 1 | CRF
DNN
DTNN | 0.779 􀀆 0.025
0.677 􀀆 0.064
0.912 􀀆 0.02 | 0.772 􀀆 0.063
0.603 􀀆 0.08
0.871 􀀆 0.022 | 0.828 􀀆 0.115
0.888 􀀆 0.11
0.949 􀀆 0.088 | 0.228 􀀆 0.081
0.163 􀀆 0.13
0.442 􀀆 0.103 | 2E-7 􀀆 0.0008 27.80 􀀆 0.2099 1877.56 􀀆 3.996 | 4.31 􀀆 0.07
82.28 􀀆 0.232
171.34 􀀆 9.362 | | 10 | CRF
DNN
DTNN | 0.932 􀀆 0.025
0.958 􀀆 0.031
0.967 􀀆 0.028 | 0.818 􀀆 0.067
0.804 􀀆 0.03
0.869 􀀆 0.033 | 0.991 􀀆 0.005
0.956 􀀆 0.045
0.950 􀀆 0.057 | 0.786 􀀆 0.027
0.801 􀀆 0.101
0.858 􀀆 0.111 | 0.01 􀀆 0.0004
31.47 􀀆 0.1487
2169.65 􀀆 3.780 | 4.3 􀀆 0.093
82.69 􀀆 0.452
185.88 􀀆 1.455 | | 100 | CRF
DNN
DTNN | 0.990 􀀆 0.004
0.994 􀀆 0.003
0.996 􀀆 0.001 | 0.901 􀀆 0.04
0.895 􀀆 0.014
0.926 􀀆 0.007 | 0.998 􀀆 0.001
0.999 􀀆 0.0002
0.993 􀀆 0.004 | 0.940 􀀆 0.021
0.976 􀀆 0.011
0.972 􀀆 0.011 | 0.02 􀀆 0.0029
74.74 􀀆 0.0955
2305.77 􀀆 11.915 | 4.39 􀀆 0.114
82.80 􀀆 0.462
186.39 􀀆 0.891 | | 1000 | CRF
DNN
DTNN | 0.999 􀀆 0.0003
0.998 􀀆 0.0002
0.999 􀀆 0.0003 | 0.933 􀀆 0.002
0.921 􀀆 0.009
0.937 􀀆 0.003 | 1.000 􀀆 5.4E-05 0.999 􀀆 0.0002 1.000 􀀆 0.0003 | 0.993 􀀆 0.001
0.989 􀀆 0.002
0.993 􀀆 0.0016 | 0.08 􀀆 0.028
74.70 􀀆 0.091
2704.20 􀀆 8.487 | 4.26 􀀆 0.084
82.74 􀀆 0.29
186.17 􀀆 0.591 | | 10000 | CRF
DNN
DTNN | 0.999 􀀆 4.36E-5
0.998 􀀆 0.0003
1.000 􀀆 1.52E-5 | 0.943 􀀆 0.002
0.927 􀀆 0.007
0.961 􀀆 0.002 | 1.000 􀀆 5. 1E-9
1.000 􀀆 0.0003
1.000 􀀆 1.32E-5 | 0.996 􀀆 0.0003
0.989 􀀆 0.0017
0.998 􀀆 9.3E-5 | 4.32 􀀆 1.2473
132.66 􀀆 0.058
4032.86 􀀆 7.79 | 4.42 􀀆 0.088
83.11 􀀆 0.258
185.42 􀀆 0.884 |

FAQs

How do processing times relate to the model evaluations?

Processing time is provided for both training and testing, allowing comparison of accuracy changes alongside computational cost for different model configurations.

Sep 3, 2026

Upload by Valentina