USC CSCI 467 - Intro to Machine Learning - Final Exam - December 7, 2023

Aug 30, 2026

Final examination for USC CSCI 467, Intro to Machine Learning (Fall 2023), held on December 7, 2023 (2:00–4:00pm) with instructor Robin Jia. The exam contains seven questions totaling 150 points, focused on foundational machine learning concepts including k-means clustering and global optimality, transformer self-attention and multi-headed attention behavior, and linear models with PCA for high-dimensional medical data. Students are instructed to answer in provided spaces and may use scratch work only on page backs.

LanguageEnglish

CategoryExam

FormatPDF

Pages20

Reading Time50 min

AccessFree to download

Question 1: K-Means and Global Optima

(a) k-means with given initialization

(b) Proof of global minimum for k=2

(d) Convergence-guaranteed vs not guaranteed methods

Question 2: Transformers and Multi-Headed Attention

(a) Encoder self-attention output for w2

(b) Decoder self-attention output for w3

(d) Removing multi-headed attention impact

(e) Reason 12 layers/12 heads is preferable

Question 3: Linear Models and PCA

(a) Classification type

(b) Why d much larger than n causes problems

(d) PCA introduces additional hyperpar

| USC CSCI 467
Intro to Machine Learning | Final Exam
December 7, 2023, 2:00-4:00pm | Fall 2023 Instructor: Robin Jia | | --- | --- | --- | Name: [USC e-mail: ](USC e-mail: @usc.edu)[ ](USC e-mail: @usc.edu)[ @usc.edu](USC e-mail: @usc.edu) [Answer the questions in the spaces provided.](Answer the questions in the spaces provided. If you run out of space)[ If you run out of space](Answer the questions in the spaces provided. If you run out of space), [continue your work](continue your work)[ ](continue your work)[on the last two pages](on the last two pages), [and indicate that your answer is there](and indicate that your answer is there). You may use the backs of pages for scratch work only. Please use pen for ease of grading. This exam has 7 questions, for a total of 150 points. This content is protected and may not be shared, uploaded, or distributed. Page 1 of 20 Question 1: K-Means and Global Optima (26 points) Recall the k-means objective function is n L (z1:n,µ1:k) =X 􀀍 x (i) − µzi 􀀍 i=1 2 , where z1:n are the cluster assignments, µ 1:k are the corresponding cluster centroids, and x (i) is the i-th input datapoint. Now consider the following dataset consisting of three datapoints ina 2-D plane: {(0, 3) ,(0 , 0) ,(4 , 0)} . (a) (6 points) Suppose you run k-means with k = 2 and have the initial cluster centers be µ 1 = (4 , 0) and µ2 = (0 , 3) . What will be the final cluster centers when k-means converges? Show your work and explain how you know that k-means has converged. (b) (6 points) Prove that the above initialization converges to the global minimum for k = 2 .(Hint: Enumerate over all possible cluster assignments and compute the objective for each assignment.) This content is protected and may not be shared, uploaded, or distributed. Page 2 of 20 (c) (6 points) Suppose we pick two distinct datapoints (out of the three datapoints) uniformly at random as the cluster centers at initialization. Will k-means always converge to the global minimum on this dataset? Justify your answer. This content is protected and may not be shared, uploaded, or distributed. Page 3 of 20 (d) For this question, you may not use k-means as an answer for either part. i. (4 points) Name one machine learning method from class (not including k-means) that is guaranteed to converge to the global optimum of its objective function. Explain why it is guaranteed. ii. (4 points) Name one machine learning method from class (not including k-means) that is not guaranteed to converge to the global optimum of its objective function. Explain why it is not guaranteed. This content is protected and may not be shared, uploaded, or distributed. Page 4 of 20 Question 2: Transformers and Multi-Headed Attention (24 points) Consider a Transformer model with a single 1-headed self-attention layer. As input, we pass in four words, each represented with an embedding of dimension 2 . That is, the inputs are vectors w 1 , w2 , w3 , w4 ∈ R2 . The single headed self-attention layer is parameterized by matrices K, Q, V ∈ R2 ×2 . For each wi, the key, query, and value vectors are defined as ki = Kwi , qi = Qwi, and vi = V wi , respectively. For simplicity, we provide the following values: q1 = 􀀔37􀀕 k 1 = 􀀔 112􀀕 v 1 = 􀀔 126􀀕 q2 = 􀀔10􀀕 k2 = 􀀔 112􀀕 v2 = 􀀔93􀀕 q3 = 􀀔02􀀕 k3 = 􀀔 112􀀕 v3 = 􀀔36􀀕 q4 = 􀀔25􀀕 k4 = 􀀔126􀀕 v4 = 􀀔41􀀕 (Note: The values here are created for ease of calculation. Technically, these values are not possible, since k 1 , k2 , k3 are identical but q1 , q2 , q3 are not.) (a) (6 points) Assume this transformer is an encoder. Calculate the output of self-attention for w2 . Note that your answer should be a vector ∈ R2 . (b) (6 points) Now assume this transformer is a decoder. Calculate the output of self-attention for w3 . Again, note that your answer should be a vector ∈ R2 . This content is protected and may not be shared, uploaded, or distributed. Page 5 of 20 (c) (4 points) During lectu

FAQs

What topics are covered in Question 1 about k-means?

Question 1 covers k-means convergence with a specific initialization, proving convergence to the global minimum for k=2, the outcome of random center initialization, and comparing optimization-convergent vs non-convergent learning methods from class.

How does the exam ask you to compute transformer self-attention?

It provides specific vectors and parameter matrices and asks for the self-attention output vector for a given word in both an encoder setting (for w2) and a decoder setting (for w3).

Why is PCA included in the linear models question?

PCA is used to address the issue where the feature dimension d is much larger than the number of training examples n, by reducing dimensionality and mitigating overfitting or computational/identifiability problems.

Aug 30, 2026

Uploaded by wps_ap_test_251126_0180