Winter School on Deep Learning

Organised by

Electronics and Communication Sciences Unit

Indian Statistical Institute

Course-End Assessment (Sol.)

Date: March 09, 2025 Duration; 1hr 10min Points: 33

Name:

Email ID: Registration ID:

Instruction: The test is comprised of 33 questions.

1. This assessment will contain a total of 33 questions.

2. Each rightly answered questions will award you with 1 point. There is

no negative marking.

3. Multiple-correct-choice questions require you to identify every correct

answer. Incomplete or incorrect selections will result in zero points.

4. Requests for additional time will not be entertained.

5. Participants are strictly prohibited from using any generative agent.

6. During the Assessment, Queries should be posted as Comment in the

Google Classroom.

7. It is preferable that Participants join the zoom meeting link during the

assessment and keep their video on.

Page Points Score

1 3

2 5

3 6

4 4

5 10

6 5

Total: 33

1. What happens when you call model.eval() in PyTorch?

A. Dropout layer behaves diﬀerently.

B. The model’s parameters become immutable.

C. Batch-normalization layer behaves diﬀerently.

D. Stops computing the gradient.

2. In PyTorch, why might you use detach() on a tensor?

✓ To prevent gradients from propagating through the tensor in the computation

graph.

 To transfer the tensor from GPU memory to main memory.

 To remove NaN values from the tensor.

 To convert the tensor into a NumPy array.

3. If the activation functions are to be arranged in the decreasing order of how much vanishing gradient

problem is likely to be most prevalent (most problems ﬁrst, least problems last), then the ordering would

be: A. Sigmoid, tanh, ReLU. B. tanh, Sigmoid, ReLU. C. ReLU, Sigmoid, tanh.

D. ReLU, tanh, Sigmoid.

4. Consider a many-to-one RNN (see the diagram). Which of the following statements are not true?

· · ·

✓ Such an architecture is suitable for machine translation.

 Other than the last cell which outputs, each of the cells use the same set of parameters.

✓ If the length of an input sequence is 7, then this architecture has 7 layers.

 Vanishing gradient problem is a likely problem for such RNN architectures if the sequences are

long.

5. Consider the relevance judgment (shown below) for a ranked list of documents for a particular query.

Assume that there are exactly 6 relevant documents for this query.

The relevance judgement for a ranked list of documents:

Rank 1 2 3 4 5 6 7 8 9 10

Relevant (Y/N) N Y N N Y Y Y Y N Y

The precision at recall 0.5 for this ranking is:

/5. B.

/5. C.

/7. D.

. E. No of the above.

6. In a standard Knowledge Distillation (KD) setup, which is(are) of the following statement(s) is(are)

true?

✓ Calculating distillation loss between the logits of both teacher and student net-

works is an example of Response-based distillation.

 Calculating distillation loss between the logits of both teacher and student networks is an

example of Feature-based distillation.

 Calculating distillation loss between the hidden features of both teacher and student networks

is an example of Response-based distillation.

✓ Calculating distillation loss between the hidden features of both teacher and stu-

dent networks is an example of Feature-based distillation.

7. In a Vision Transformer (ViT), what is the purpose of the positional embeddings?

✓ To introduce spatial information that is lost when images are divided into patches.

 To reduce the computational complexity of self-attention.

 To enhance feature extraction using convolutional ﬁlters.

 To normalize the input patches before feeding them into the transformer.

8. In a Vision Transformer (ViT), how does the self-attention mechanism aﬀect feature representation

compared to Convolutional Neural Networks (CNNs)?

A. Self-attention captures only local dependencies, similar to convolutional kernels.

B. Self-attention enables global receptive ﬁelds from the ﬁrst layer, unlike CNNs.

C. Self-attention reduces the number of parameters compared to CNNs for the same task.

Points earned: out of a possible 5 points

D. Self-attention operates independently on each image patch without considering other patches.

9. Consider a 3 × 3 matrix A whose eigenvalues are 2, −2 and − 1. Then Trace of (A

+ A

) is:

A. 4. B. 8. C. 16. D. 2. E. Missing Information.

10. Which of the following represents the Betti numbers (β

, β

) of the given ﬁgure (A Hollow Torus)?

A. (2, 1, 0). B. (1, 2, 1). C. (1, 1, 0). D. (0, 2, 1).

11. Let A, B be 2 × 2 matrices entries from Real numbers such that det(A) = det(B) and trace(A) =

trace(B). Then, which among the following is an INCORRECT Option?

A. A and B have the same set of eigenvalues.

B. A and B need NOT be similar.

C. A and B are similar.

D. None of these.

12. Which of the following statements are true regarding the role of the autograd feature in PyTorch?

(Select all that apply)

✓ Autograd only works with tensors which are initialised with requires

grad=True.

✓ Autograd can be disabled by setting torch.no grad(), reducing memory consump-

tion and computation during inference.

✓ Autograd tracks only the operations performed on tensors, and the gradients are

computed only when backward() is called.

 Autograd is only available in PyTorch and cannot be replicated in other frameworks like

TensorFlow or JAX.

13. In the context of training deep learning models, which of the following statements are correct about

optimizers and the learning rate? (Select all that apply)

✓ The learning rate controls the size of the step the optimizer takes during each

update of the model’s parameters.

✓ A large learning rate can lead to faster convergence, but it may risk overshooting

the minimum of the loss function.

 The learning rate should remain constant throughout training to ensure stable convergence.

✓ Learning rate schedules, such as reducing the learning rate after a ﬁxed number

of epochs, can help achieve better convergence in some cases.

14. Suppose we need to perform a shape-preserving operation op (say, sigmoid) after application of each

encoder layer of a transformer encoder. Which of the following codes correctly realises such a modiﬁca-

tion?

Points earned: out of a possible 6 points

A. class CustomEncoderLayer(nn.TransformerEncoderLayer):

def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1, activation="relu"):

super().__init__(d_model, nhead, dim_feedforward, dropout, activation)

self.op = nn.Sigmoid()

def forward(self, src, src_mask = None, src_key_padding_mask = None, is_causal = False):

self.forward(src)

return self.op(encret)

B. class CustomEncoderLayer(nn.Module):

def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1, activation="relu"):

self.op = nn.Sigmoid()

def forward(self, src, src_mask = None, src_key_padding_mask = None, is_causal = False):

encret = super().forward(src, src_mask, src_key_padding_mask, is_causal)

return self.op(encret)

C. class CustomEncoderLayer(nn.TransformerEncoderLayer):

def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1, activation="relu"):

super().__init__(d_model, nhead, dim_feedforward, dropout, activation)

self.op = nn.Sigmoid()

def forward(self, src, src_mask = None, src_key_padding_mask = None, is_causal = False):

encret = super().forward(src, src_mask, src_key_padding_mask, is_causal)

return self.op(encret)

D. class CustomEncoderLayer(nn.Module):

def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1, activation="relu"):

super().__init__(d_model, nhead, dim_feedforward, dropout, activation)

self.op = nn.Sigmoid()

def forward(self, src, src_mask = None, src_key_padding_mask = None, is_causal = False):

encret = super().forward(src, src_mask, src_key_padding_mask, is_causal)

return self.op(encret)

15. Consider an integer c ∈ [−2

, 2

− 1], we propose a modiﬁed binary representation of c, mbin(c) ∈

{0, 1}

p+1

as follows. The ﬁrst bit (Most Signiﬁcant Bit) in bin(c) is 0 (or 1) if c is non-negative (or,

negative). The following p bits are binary representation of |c| (prepended with 0’s, if necessary). For

example, consider p = 6 and c = −13; then mbin(−13) = 1001101. The ﬁrst bit 1 stands for negative

be accessed using the notation mbin(c)[i] where i ∈ {0, 1, . . . , p}. Thus mbin(−13)[4] is 0.

Now, we need to implement the activation ReLU involving only logical operators (¬, ∨, ∧). Here, the

operators act element-wise, e.g. ¬101 = 010 and ¬101 ∨ 010 = 010. The which of the following is/are

true for ReLU(c) in mbin system:

 (¬ mbin(c)[0])

p+1

∨ ¬ mbin(c).

✓ (¬ mbin(c)[p])

p+1

∧ mbin(c).

 (¬ mbin(c)[0])

p+1

∨ mbin(c + 1).

 (¬ mbin(c)[0])

p+1

∧ mbin(c + 1).

16. Which of the following option(s) is(are) correct?

 GCN assigns unequal weights to the edges during performing neighborhood aggregation.

✓ GCN assigns equal weights to the edges during performing neighborhood aggre-

gation.

✓ Symmetric normalization of adjacency matrix is performed in GCN to avoid nu-

merical instability.

✓ Symmetric normalization of adjacency matrix is performed in GCN to make faster

the training convergence.

17. Which of these methods do not require a reward model?

A. PPO B. DPO C. GRPO D. None of the above.

18. Which of the following is not a tokenizer? Check all that applies

✓ VerbPiece  UniGram. ✓ NGrams.  SentencePiece.  WordPiece.

Points earned: out of a possible 4 points

19. In a typical self-supervised SimClr setting given a batch of n samples how many positive and negative

pairs can be generated?

A. n positive pairs and 2n(n − 2) negative pairs.

B. n positive pairs and 4n(n − 1) negative pairs.

C. n − 1 positive pairs and 4n

negative pairs.

D. n positive pairs and 2n(n − 1) negative pairs.

E. n − 1 positive pairs and 4n(n − 2) negative pairs.

20. Which architecture can be trained end-to-end for object detection without any multiple phases among

the following?

A. R-CNN. B. Fast R-CNN. C. Faster R-CNN. D. YOLO. E. Mask R-CNN.

21. Select the odd one out. A. V-Net. B. U-Net++. C. 3D U-Net. D. U-NETR. E. SegNet.

22. Which of the following is the main splitting criterion for the CART algorithm?

A. Gain Ratio. B. Information gain. C. Divergence Gain. D. Gini impurity.

23. A real-valued 2 × 2 matrix has trace 3 and determinant is −10. Then which of the following pair

represents its eigenvalues.

A. (10, 3) B. (0, 0) C. (5, −2) D. More info required.

24. What is the role of the bottleneck layer in an autoencoder?

 To increase the dimensionality of the input data.

✓ To encode the most important features of the input data.

 To reconstruct the input data with higher accuracy.

 To perform classiﬁcation tasks.

25. Why does Reptile usually require less memory compared to MAML?

A. Reptile usually uses smaller batch sizes.

B. MAML is diﬃcult to implement eﬃciently, leading to higher memory utilization.

C. Reptile only uses ﬁrst order gradients, unlike MAML which relies on second order

gradients.

D. Reptile uses smaller networks than MAML.

26. Consider a partially labeled graph, a k-layered GCN is applied. Then which is(are) of the following

statement(s) is(are) true?

 At each layer, both node features and edge connectivity of the input graph will alter.

✓ At each layer, ony node features of the input graph will alter.

✓ GCN will aggregate node features form k-hop neighborhoods.

 GCN will aggregate node features form (k+1)-hop neighborhoods.

27. Consider two bi-variate functions f(x, y) = 2x

+ y, g(x, y) = −4x + 2y

. Compute Jacobian matrix by

diﬀerentiating [f, g] w.r.t. [x, y] and ﬁnd its determinant at x = 1, y = −1.

A. 0. B. 12. C. 20. D. −12.

28. The main diﬀerence between Variational Autoencoders (VAE) and traditional autoencoders is:

A. VAEs use nonlinear transformations, whereas traditional autoencoders use linear transforma-

tions.

Points earned: out of a possible 10 points

B. VAEs introduce a probability distribution in the latent space between the encoder

and decoder, which traditional autoencoders do not.

C. VAEs are only applicable to image data, while traditional autoencoders can be applied to any

type of data.

D. The loss function of VAEs includes reconstruction loss, whereas that of traditional autoencoders

does not.

29. In score-based generative models and diﬀusion models, the score function is deﬁned as the gradient of

the log probability density with respect to the input. If we have a distribution p

(x) at time t, which of

the following correctly represents the score function?

A. ∇

log p

(x). B. ∇

log p

(x). C. ∇

(x). D.

(x)

∇

(x)

30. What is the computational complexity of a single GCN operation?

A. O(n

). B. O(n + m). C. O(m). D. O(nm).

31. In the ϵ-greedy policy, what is the probability of selecting the greedy action, out of N actions?

A. ϵ. B. 1 − ϵ. C. 1 −

. D. 1 − ϵ +

32. Why is REINFORCE with baseline used instead of standard REINFORCE in policy gradient methods?

A. To reduce variance in gradient updates without changing the expected value.

B. To increase the learning rate dynamically during training.

C. To completely eliminate randomness in action selection.

D. To ensure the policy converges to the global optimum in fewer steps.

33. How does the bias-variance tradeoﬀ diﬀer between Monte Carlo (MC) methods and Bootstrapping

methods in Reinforcement Learning?

A. Monte Carlo has higher variance but lower bias, while Bootstrapping has lower

variance but higher bias.

B. Monte Carlo has higher bias but lower variance, while Bootstrapping has lower bias but higher

variance.

C. Both Monte Carlo and Bootstrapping have low bias and high variance.

D. Both Monte Carlo and Bootstrapping have low variance and high bias.

Points earned: out of a possible 5 points