CSCC11H - Machine Learning and Data Mining - December 2014 University of Toronto at Scarborough Exam

Aug 30, 2026

Upload by Ophelia

December 2014 CSCC11H exam (University of Toronto at Scarborough) for Machine Learning and Data Mining, Duration 3 hours with no aids allowed. The paper assesses core concepts in ensemble learning, Naive Bayes assumptions, K-fold cross-validation, gradient descent variants, expected loss in stochastic optimization, and probability rules leading to Bayes’ Rule. It also evaluates basis function regression using likelihood and closed-form solutions, Bayesian weight posteriors with Gaussian priors, supervised classification model choice and decision boundaries, and PCA including objective functions and algorithmic steps.

LanguageEnglish

CategoryExam

FormatPDF

Pages16

Reading Time40 min

AccessFree to download

CSCC11H: Machine Learning and Data Mining

Short Answer (1)

Basis Function Regression (2)

Classification (3)

PCA (4)

UNIVERSITY OF TORONTO AT SCARBOROUGH Department of Computer and Mathematical Sciences DECEMBER 2014 EXAMINATIONS CSCC11H: Machine Learning and Data Mining Duration: 3 hours Aids allowed: None There are 16 pages total (including this page) Please answer questions on the exam pages in the space provided. You may use the back of the pages for scratch work but ﬁnal answers should be in the provided spaces. Read questions carefully and answer as neatly, clearly and concisely as possible; illegible or incomprehensible answers will get no marks. Should a question be unclear or ambiguous, make a reasonable interpretation and state what you have assumed before answering. Partial credit will be given for clear formulations of how to solve the problems. Family name: Given names: Student number: Signature : By signing above, I certify that the work contained within is my work and my work alone and understand that copying another students work or allowing another student to copy my work is a serious academic offence. Question Marks 1 / 23 2 / 15 3 / 12 4 / 20 5 / 21 6 / 9 Bonus Total / 100 1 Short Answer [23 marks] (a) [2 marks] Ensemble methods combine together many simple, poorly performing classiﬁersin order to produce a single, high quality classiﬁer. Name two ensemble methods. (b) [2 marks] What is the principal assumption in the Naive Bayes' model, and when is this assumption useful? (c) [4 marks] Describe the K-fold cross-validation algorithm for model selection. Question 1 continued. (d) [1 marks] Gradient Descent and Stochastic Gradient Descent are very similar in that they both iteratively update the parameter vector based on the gradient of a function. Describe either in words how they different. (e) [2 marks] Stochastic Gradient Descent is motivated by the idea that there exists (in the universe) an inﬁnite amount of data which is distributed according to p(x; y) . Then, given some parameters 􀀒 and a loss function L (x; yj􀀒) we wish to minimize the expected loss. Deﬁne mathematically the expected loss. Question 1 continued. (f) [6 marks] In class we discussed three different kinds of Unsupervised Learning problems. List the three types of problems and for each name a method which addresses that problem. (g) [3 marks] What is the Product Rule of probability? Deﬁne it and use it to derive Bayes' Rule. (h) [3 marks] Use Bayes' Rule to write the probability of the parameters 􀀒 given the data D. Specify which elements correspond to the posterior, the likelihood, the prior and the evidence. 2 Basis Function Regression [15 marks] Suppose you are given a dataset of N training pairs f (xi ; yi)g, such that xi 2 R; yi 2 R. We want to ﬁt the following model to the data: K y = f (x) + n; where f (x) = Xwj sin(cx + dj) ; and n 􀀘 N (0 ; 1=􀀋) : (1) j=1 Model parameters include the weights w = [w1 ; :::wK ]T , the frequency c, the phase shifts d =[dj :::dK ], and the variance 1=􀀋 . (a) [2 marks] Provide an analytic expression for p(yj x; c; w; 􀀋; d) . (b) [2 marks] Formulate the likelihood of the training data given this model, p(yj x; c; w; 􀀋; d) . Simplify where possible and include all terms and constants. (c) [3 marks] Write the negative log-likelihood, and simplify as much as possible. Include all terms and constants. To simplify the remainder of the question, it is recommended that you write the summation in terms of a dot product (i.e., wTb) . Question 2 continued. (d) [4 marks] Suppose all parameters but w are known. Give a closed-form expression for the optimal weight vector w that maximizes the data likelihood. Question 2 continued. (e) [4 marks] Suppose we assume a Gaussian prior on the weights w 􀀘 N (0; 􀀌I) . Derive the posterior distribution over the weights w, given c, 􀀋, and d. What type of distribution is this posterior? 3 Classiﬁcation [12 marks] Consider the following classiﬁcation algorithms: Gaussian Class Conditional, K-Nearest

FAQs

What topics are covered in the Short Answer section of CSCC11H?

It covers ensemble methods, Naive Bayes’ principal assumption, K-fold cross-validation, differences between gradient descent variants, expected loss in stochastic optimization, unsupervised learning problem types, and probability rules leading to Bayes’ Rule.

How does the Basis Function Regression question model the data and what are the key parameters?

It models outputs as y = f(x) + n with f(x)=Σ wj sin(cx + dj) and noise n ~ N(0, 1/λ). Parameters include weights w, frequency c, phase shifts d, and noise variance 1/λ.

What does the PCA section ask students to demonstrate?

It asks for the relationship between latent coordinates and observed data, the objective minimized to learn PCA parameters, distinctions between probabilistic PCA and regular PCA, and the steps of the PCA algorithm.

Aug 30, 2026

Upload by Ophelia