class: center, middle, inverse, title-slide .title[ # Classification of the Residues after High and Low Order Explosions ] .subtitle[ ## using Machine Learning Techniques on Fourier Transform Infrared (FTIR) spectra ] .author[ ### Krzysztof Banas<br><span style="font-size: 80%;">SSLS<br></span> ] .date[ ### 2023-03-30 ] --- class: inverse center middle # Introduction --- ## SPF Project .pull-left[ * in collaboration with SPF * high and low order explosions * debris collected * analysed with infrared spectroscopy: FTIR spectrometer ] .pull-right[  ] --- ## Experiments .pull-left[ - KBr pellets - mid-IR range - multiple spectra measured - spectral signatures from high energetic material - ... but also from the surface material (paper, plastic, textile) ] .pull-right[ ]  --- ## Motivation for this sub-project .pull-left[ * check which materials are more suitable as sample collectors * check what is the influence of the order of explosion on the classification * investigate dimension reduction method in combination with discrimination * building classification model: combined PCA-LDA ] .pull-right[  ] --- class: inverse center middle # Results --- ## Spectral Data .pull-left[ * typically more variables than observations * variables are highly correlated and auto-correlated * before modeling need for dimension reduction * pre-processing is crucial ] .pull-right[  ] --- ## Pre-processing .pull-left[  ] .pull-right[  ] --- class: inverse center middle ## Dimension Reduction --- ## How PCA works ] --- ## How PCA works ] --- ## PCA on decathlon dataset <div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-beba10c0e7007f2b5fba" style="width:100%;height:auto;"></div> <script type="application/json" data-for="htmlwidget-beba10c0e7007f2b5fba">{"x":{"filter":"none","vertical":false,"data":[["SEBRLE","CLAY","KARPOV","BERNARD","YURKOV","WARNERS","ZSIVOCZKY","McMULLEN","MARTINEAU","HERNU","BARRAS","NOOL","BOURGUIGNON","Sebrle","Clay","Karpov","Macey","Warners","Zsivoczky","Hernu","Nool","Bernard","Schwarzl","Pogorelov","Schoenbeck","Barras","Smith","Averyanov","Ojaniemi","Smirnov","Qi","Drews","Parkhomenko","Terek","Gomez","Turi","Lorenzo","Karlivans","Korkizoglou","Uldal","Casarsa"],[11.04,10.76,11.02,11.02,11.34,11.11,11.13,10.83,11.64,11.37,11.33,11.33,11.36,10.85,10.44,10.5,10.89,10.62,10.91,10.97,10.8,10.69,10.98,10.95,10.9,11.14,10.85,10.55,10.68,10.89,11.06,10.87,11.14,10.92,11.08,11.08,11.1,11.33,10.86,11.23,11.36],[7.58,7.4,7.3,7.23,7.09,7.6,7.3,7.31,6.81,7.56,6.97,7.27,6.8,7.84,7.96,7.81,7.47,7.74,7.14,7.19,7.53,7.48,7.49,7.31,7.3,6.99,6.81,7.34,7.5,7.07,7.34,7.38,6.61,6.94,7.26,6.91,7.03,7.26,7.07,6.99,6.68],[14.83,14.26,14.77,14.25,15.19,14.31,13.48,13.76,14.57,14.41,14.09,12.68,13.46,16.36,15.23,15.93,15.73,14.48,15.31,14.65,14.26,14.8,14.01,15.1,14.77,14.91,15.24,14.44,14.97,13.88,13.55,13.07,15.69,15.15,14.57,13.62,13.22,13.3,14.81,13.53,14.92],[2.07,1.86,2.04,1.92,2.1,1.98,2.01,2.13,1.95,1.86,1.95,1.98,1.86,2.12,2.06,2.09,2.15,1.97,2.12,2.03,1.88,2.12,1.94,2.06,1.88,1.94,1.91,1.94,1.94,1.94,1.97,1.88,2.03,1.94,1.85,2.03,1.85,1.97,1.94,1.85,1.94],[49.81,49.37,48.37,48.93,50.42,48.68,48.62,49.91,50.14,51.1,49.48,49.2,51.16,48.36,49.19,46.81,48.97,47.97,49.4,48.73,48.81,49.13,49.76,50.79,50.3,49.41,49.27,49.72,49.12,49.11,49.65,48.51,51.04,49.56,48.61,51.67,49.34,50.54,51.16,50.95,53.2],[14.69,14.05,14.09,14.99,15.31,14.23,14.17,14.38,14.93,15.06,14.48,15.29,15.67,14.05,14.13,13.97,14.56,14.01,14.95,14.25,14.8,14.17,14.25,14.21,14.34,14.37,14.01,14.39,15.01,14.77,14.78,14.01,14.88,15.12,14.41,14.26,15.38,14.98,14.96,15.09,15.39],[43.75,50.72,48.95,40.87,46.26,41.1,45.67,44.41,47.6,44.99,42.1,37.92,40.49,48.72,50.11,51.65,48.34,43.73,45.62,44.72,42.05,44.75,42.43,44.6,44.41,44.83,49.02,39.88,40.35,42.47,45.13,40.11,41.9,45.62,40.95,39.83,40.22,43.34,46.07,43.01,48.66],[5.02,4.92,4.92,5.32,4.72,4.92,4.42,4.42,4.92,4.82,4.72,4.62,5.02,5,4.9,4.6,4.4,4.9,4.7,4.8,5.4,4.4,5.1,5,5,4.6,4.2,4.8,4.6,4.7,4.5,5,4.8,5.3,4.4,4.8,4.5,4.5,4.7,4.5,4.4],[63.19,60.15,50.31,62.77,63.44,51.77,55.37,56.37,52.33,57.19,55.4,57.44,54.68,70.52,69.71,55.54,58.46,55.39,63.45,57.76,61.33,55.27,56.32,53.45,60.89,64.55,61.52,54.51,59.26,60.88,60.79,51.53,65.82,50.62,60.71,59.34,58.36,52.92,53.05,60,58.62],[291.7,301.5,300.2,280.1,276.4,278.1,268,285.1,262.1,285.1,282,266.6,291.7,280.01,282,278.11,265.42,278.05,269.54,264.35,276.33,276.31,273.56,287.63,278.82,267.09,272.74,271.02,275.71,263.31,272.63,274.21,277.94,290.36,269.7,290.01,263.08,278.67,317,281.7,296.12],[1,2,3,4,5,6,7,8,9,10,11,12,13,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28],[8217,8122,8099,8067,8036,8030,8004,7995,7802,7733,7708,7651,7313,8893,8820,8725,8414,8343,8287,8237,8235,8225,8102,8084,8077,8067,8023,8021,8006,7993,7934,7926,7918,7893,7865,7708,7592,7583,7573,7495,7404],["Decastar","Decastar","Decastar","Decastar","Decastar","Decastar","Decastar","Decastar","Decastar","Decastar","Decastar","Decastar","Decastar","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG","OlympicG"]],"container":"<table class=\"display\">\n <thead>\n <tr>\n <th> <\/th>\n <th>100m<\/th>\n <th>Long.jump<\/th>\n <th>Shot.put<\/th>\n <th>High.jump<\/th>\n <th>400m<\/th>\n <th>110m.hurdle<\/th>\n <th>Discus<\/th>\n <th>Pole.vault<\/th>\n <th>Javeline<\/th>\n <th>1500m<\/th>\n <th>Rank<\/th>\n <th>Points<\/th>\n <th>Competition<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"columnDefs":[{"className":"dt-right","targets":[1,2,3,4,5,6,7,8,9,10,11,12]},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false}},"evals":[],"jsHooks":[]}</script> --- ## Correlation between variables <img src="index_files/figure-html/unnamed-chunk-2-1.png" width="100%" /> --- ## Principal Components: screeplot and scoreplot <div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-cbcf73d83e4629c5eea6" style="width:100%;height:auto;"></div> <script type="application/json" data-for="htmlwidget-cbcf73d83e4629c5eea6">{"x":{"filter":"none","vertical":false,"data":[["comp 1","comp 2","comp 3","comp 4","comp 5","comp 6","comp 7","comp 8","comp 9","comp 10"],[" 3.114"," 2.027"," 1.390"," 1.321"," 0.861"," 0.566"," 0.497"," 0.108"," 0.094"," 0.023"],[" 31.144"," 20.267"," 13.903"," 13.206"," 8.610"," 5.656"," 4.966"," 1.079"," 0.940"," 0.229"],[" 31.144"," 51.411"," 65.314"," 78.520"," 87.129"," 92.785"," 97.751"," 98.831"," 99.771","100.000"]],"container":"<table class=\"display\">\n <thead>\n <tr>\n <th> <\/th>\n <th>eigenvalue<\/th>\n <th>percentage of variance<\/th>\n <th>cumulative percentage of variance<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"columnDefs":[{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false}},"evals":[],"jsHooks":[]}</script> --- <img src="index_files/figure-html/unnamed-chunk-4-1.png" width="100%" /> --- ## Correlation between a variable and a principal component <div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-0ca184cacb468872fc02" style="width:100%;height:auto;"></div> <script type="application/json" data-for="htmlwidget-0ca184cacb468872fc02">{"x":{"filter":"none","vertical":false,"data":[["100m","Long.jump","Shot.put","High.jump","400m","110m.hurdle","Discus","Pole.vault","Javeline","1500m"],["-0.8418"," 0.6538"," 0.3493"," 0.3372","-0.6051","-0.8761"," 0.5464","-0.0598"," 0.0153"," 0.5537"],["-0.0380","-0.0034"," 0.6146","-0.3867"," 0.4733"," 0.2918"," 0.2735"," 0.7556"," 0.4649"," 0.5718"],[" 0.3966","-0.4199"," 0.5218"," 0.2213"," 0.1503","-0.1621"," 0.6745","-0.2152","-0.4087","-0.1350"],["-0.0351"," 0.1667"," 0.2658"," 0.7115"," 0.2279"," 0.2433","-0.0461","-0.3165"," 0.6715","-0.2243"],[" 0.2053"," 0.0466"," 0.3355"," 0.1303","-0.5034","-0.0345","-0.2068"," 0.4944"," 0.0032","-0.3813"]],"container":"<table class=\"display\">\n <thead>\n <tr>\n <th> <\/th>\n <th>Dim.1<\/th>\n <th>Dim.2<\/th>\n <th>Dim.3<\/th>\n <th>Dim.4<\/th>\n <th>Dim.5<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"columnDefs":[{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false}},"evals":[],"jsHooks":[]}</script> --- ## Variable Correlation Plot <img src="index_files/figure-html/unnamed-chunk-6-1.png" width="100%" /> --- ## Quality of Representation (cos²) <div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-b0f305a0e1b80c82c83c" style="width:100%;height:auto;"></div> <script type="application/json" data-for="htmlwidget-b0f305a0e1b80c82c83c">{"x":{"filter":"none","vertical":false,"data":[["100m","Long.jump","Shot.put","High.jump","400m","110m.hurdle","Discus","Pole.vault","Javeline","1500m"],["0.70865067","0.42741078","0.12203430","0.11372115","0.36615331","0.76751118","0.29853043","0.00357099","0.00023293","0.30660630"],["0.00144102","0.00001160","0.37771968","0.14955897","0.22401258","0.08514249","0.07480675","0.57089076","0.21611091","0.32699009"],["0.15728916","0.17634947","0.27227398","0.04898328","0.02258608","0.02628903","0.45495546","0.04629517","0.16701619","0.01823668"],["0.00122876","0.02778479","0.07066813","0.50625000","0.05195970","0.05919477","0.00212219","0.10018061","0.45086976","0.05031061"],["0.04214812","0.00217577","0.11253218","0.01697411","0.25338787","0.00118725","0.04275028","0.24444574","0.00000997","0.14535524"]],"container":"<table class=\"display\">\n <thead>\n <tr>\n <th> <\/th>\n <th>Dim.1<\/th>\n <th>Dim.2<\/th>\n <th>Dim.3<\/th>\n <th>Dim.4<\/th>\n <th>Dim.5<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"columnDefs":[{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false}},"evals":[],"jsHooks":[]}</script> --- ## Correlation Plot .pull-left[ <img src="index_files/figure-html/unnamed-chunk-8-1.png" width="100%" /> ] .pull-right[ <img src="index_files/figure-html/unnamed-chunk-9-1.png" width="100%" /> ] --- ## Contributions of variables to PC1 <img src="index_files/figure-html/unnamed-chunk-10-1.png" width="100%" /> --- ## Contributions of variables to PC2 <img src="index_files/figure-html/unnamed-chunk-11-1.png" width="100%" /> --- ## Contributions of variables to PC1 and PC2 <img src="index_files/figure-html/unnamed-chunk-12-1.png" width="100%" /> --- ## Principal Components: scoreplot <img src="index_files/figure-html/unnamed-chunk-13-1.png" width="100%" /> --- ## PCA: Practical Applications - Finance: analyze stock market data to identify patterns - Image compression: compress digital images - Genetics: analyze genetic data to identify patterns - Social media: social media data to identify trends in user behavior and preferences - Marketing: consumer data to identify patterns in purchasing behavior and preferences - Healthcare: patient data to identify risk factors and predict outcomes - Quality control: manufacturing data to identify patterns in product quality and performance - Agriculture: crop data to identify patterns in yields and optimize farming practices - Climate research: climate data to identify patterns in weather patterns and predict future climate trends - Sports: player data to identify key factors that contribute to winning games and championships. --- ## PCA for spectrosopic data .pull-left[ - dimension reduction - removing correlation between variables - finding patters - unsupervised method - *lossy technique* - *scaling variables can yield different results* - *problems with interpretation: PCs are linear combination of original features* ] .pull-right[ ] --- ## Spectral dataset <div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-44c5a3b3adca03fef24c" style="width:100%;height:auto;"></div> <script type="application/json" data-for="htmlwidget-44c5a3b3adca03fef24c">{"x":{"filter":"none","vertical":false,"data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100","101","102","103","104","105","106","107","108","109","110","111","112","113","114","115","116","117","118","119","120","121","122","123","124","125","126","127","128","129","130","131","132","133","134","135","136","137","138","139","140","141","142","143","144","145","146","147","148","149","150","151","152","153","154","155","156","157","158","159","160","161","162","163","164","165","166","167","168","169","170","171","172","173","174","175","176","177","178","179","180","181","182","183","184","185","186","187","188","189","190","191","192","193","194","195","196","197","198","199","200","201","202","203","204","205","206","207","208","209","210","211"],["CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","CE4","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""],["HIG","HIG","HIG","HIG","HIG","HIG","HIG","HIG","HIG","HIG","HIG","HIG","HIG","HIG","HIG","HIG","LOW","LOW","LOW","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""],[0.42882,0.30061,0.43598,0.48907,0.18242,0.7116,0.28766,0.29107,0.17742,0.10895,0.37,0.16518,0.60512,0.59827,0.50959,0.56813,0.02402,0.00338,0.0746,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.42765,0.29644,0.4275,0.48801,0.18303,0.71245,0.29878,0.28991,0.1816,0.10261,0.37011,0.16182,0.58663,0.59969,0.51267,0.5762,0.0343,0.00159,0.07404,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.42463,0.29359,0.4228,0.48633,0.18237,0.71552,0.30834,0.2899,0.18356,0.09856,0.36849,0.15776,0.57209,0.59837,0.51379,0.58388,0.04392,0,0.07387,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.41993,0.29049,0.42038,0.48258,0.17873,0.71944,0.31276,0.29081,0.18193,0.09685,0.36516,0.15264,0.5636,0.59487,0.51213,0.58876,0.05275,-0.00191,0.07457,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.41387,0.28594,0.41869,0.47773,0.17227,0.7227,0.31177,0.29306,0.1773,0.09674,0.36207,0.14691,0.5616,0.59095,0.50888,0.58908,0.06149,-0.00399,0.07625,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.35003,0.29019,0.42921,0.34767,0.2873,1.5349,0.16709,0.14281,0.10052,0.10942,1.07671,0.13559,0.47207,1.02394,0.31655,0.38114,0.00465,0.00444,0.00044,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.35004,0.29025,0.42926,0.34772,0.28741,1.53514,0.16707,0.14274,0.10048,0.10944,1.07675,0.13558,0.47214,1.02403,0.3166,0.38118,0.00456,0.00435,0.00043,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.35005,0.2903,0.42932,0.34777,0.2875,1.53538,0.16705,0.14268,0.10042,0.10947,1.0768,0.13557,0.47222,1.02413,0.31667,0.38121,0.00449,0.00427,0.00042,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.35005,0.29036,0.42936,0.34781,0.28758,1.53553,0.16704,0.14264,0.10036,0.1095,1.07689,0.13555,0.4723,1.02425,0.31676,0.38125,0.00442,0.00421,0.00042,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.35005,0.29041,0.42939,0.34783,0.28765,1.53556,0.16704,0.14264,0.10032,0.10955,1.07705,0.13553,0.47236,1.02438,0.31685,0.38129,0.00436,0.00417,0.00042,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.35005,0.29048,0.42941,0.34784,0.28771,1.53552,0.16704,0.14266,0.10033,0.1096,1.07731,0.13553,0.47241,1.02448,0.31691,0.38134,0.00428,0.00417,0.00042,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[0.35004,0.29055,0.42943,0.34784,0.28778,1.53548,0.16705,0.14269,0.10038,0.10965,1.0777,0.13554,0.47244,1.02457,0.31693,0.3814,0.0042,0.00418,0.00041,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null]],"container":"<table class=\"display\">\n <thead>\n <tr>\n <th> <\/th>\n <th>expl<\/th>\n <th>order<\/th>\n <th>X400.199<\/th>\n <th>X401.163<\/th>\n <th>X402.128<\/th>\n <th>X403.092<\/th>\n <th>X404.057<\/th>\n <th>.....<\/th>\n <th>X3994.28<\/th>\n <th>X3995.24<\/th>\n <th>X3996.21<\/th>\n <th>X3997.17<\/th>\n <th>X3998.13<\/th>\n <th>X3999.1<\/th>\n <th>X4000.06<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"columnDefs":[{"className":"dt-right","targets":[3,4,5,6,7,9,10,11,12,13,14,15]},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false}},"evals":[],"jsHooks":[]}</script> --- ## Loadings and contribution plot .pull-left[  ] .pull-right[ - first two PCA - all wavenumbers - colour coded contribution cos² contribution to PC1 and PC2 ] --- ## Scoreplots .pull-left[  ] .pull-right[ - first two PCA - all observations (spectra) - colour coded class membership (explosive and order) ] --- class: inverse center middle ## Identification/Classification/Grouping --- ## 2D partitions <img src="FIGURES/partition_plot.png" width="800px" style="position:absolute; left:10px; top:130px;"> --- ## Building Classification Model - first: use PCA on spectral data for Dimension Reduction - then: use LDA to build the Classification Model using a first few PC's - finally: test the Classification Model with new data --- background-image: url("FIGURES/LDA_00.png") background-position: 50% 50% background-size: 40% 60% ## How LDA works¹ .footnote[[1] Source: Linear discriminant analysis : a detailed tutorial Gaber, T, Tharwat, A, Ibrahim, A and Hassanien, AE AI Communications (2017)] --- ## Data Preparation for LDA - **Classification Problems** applied to classify the categorical output variable (suitable for binary and multi-class classification problems) - **Gaussian Distribution** The standard LDA model applies the Gaussian Distribution of the input variables. One should review the univariate distribution of each variable and transform them into more Gaussian-looking distributions (use log and root for exponential distributions and Box-Cox for skewed distributions) - **Remove Outliers** It is good to firstly remove the outliers from your data because these outliers can skew the basic statistics used to separate classes in LDA, such as the mean and the standard deviation - **Same Variance** As LDA always assumes that all the input variables have the same variance, hence it is always a better way to firstly standardize the data before implementing an LDA model. By this, the mean will be 0, and it will have a standard deviation of 1. --- class: inverse center middle ## LDA on principal components --- ## Discriminant functions plot .pull-left[  ] .pull-right[ - first two linear discriminant functions - all observations (originaly spectra) - colour coded class membership (explosive and order) ] --- ## Discriminant functions plot .pull-left[  ] .pull-right[ - first two linear discriminant functions - all observations (originaly spectra) - colour coded class membership (explosive and order) ] --- ## Comparison .pull-left[ ### PCA - unsupervised - maximize the variance in the given dataset - good performer for a comparatively small sample size ] .pull-right[ ### LDA - supervised - find the linear discriminants to represent the axes that maximize separation between different classes - more suitable for multi-class classification tasks ] --- class: inverse center middle # Conclusions --- ## Take home message - Spectroscopic data are multidimensional and highly correlated -- - PCA "transfers" most variability of the system to the first few principal components -- - PCA removes correlation between variables -- - For classification model we need another method: LDA -- - With PCs as input variables LDA works well in predicting the class membership -- - We can validate model with various statistical methods, for example Leave-One-Out --- ## Where to find: .pull-left[ ### Manuscript:  ] .pull-right[ ### Slides: https://krzbanas.github.io/2023-03-30_SSLS_Seminar  ] --- class: center, middle # Thank You! Slides created via the R packages: [**xaringan**](https://github.com/yihui/xaringan)<br> [gadenbuie/xaringanthemer](https://github.com/gadenbuie/xaringanthemer) The chakra comes from [remark.js](https://remarkjs.com), [**knitr**](http://yihui.name/knitr), and [R Markdown](https://rmarkdown.rstudio.com).