LLM foundations interview questions

Key concept: Do you know the biases and validate the evaluator itself.

Reference answer: Judges are biased: position bias (favor the first option), verbosity bias (favor longer answers), self-preference (favor their own style), and leniency. Mitigate by randomizing and averaging both orders, using a rubric with explicit criteria rather than "which is better," scoring on a defined scale with few-shot anchors, and calibrating against a human-labeled set (measure agreement with Cohen's kappa). Use a strong judge model, version the judge prompt, and spot-check disagreements. Never ship a judge you have not validated against humans - it just launders bias into a metric.

Key concept: Do you understand the sources of variance and design for them.

Reference answer: Some variance is inherent (sampling, floating-point non-associativity across batches and GPUs, MoE routing, provider model updates). Reduce it where you need stability: temperature 0 / greedy, fixed seeds where supported, pin the exact model snapshot (not a floating alias), and constrain output format. For evaluation, sample N times and report the distribution (pass@k, variance) rather than judging one roll. For features that must be reproducible (audits, caching), cache the output keyed on the exact input. Accept you cannot get bit-exact determinism from a hosted model and design around it.

Key concept: Do you know long context is not uniformly usable.

Reference answer: LLMs attend most strongly to the start and end of the context window; information buried in the middle is recalled less reliably. So put the most important instructions and the most relevant retrieved chunks near the top or bottom, not the middle; do not assume a 128k window lets you dump everything and trust recall; rerank so the best chunks land in high-attention positions; and keep context lean - more tokens often means slower and worse, not better. Test recall at length rather than assuming the advertised window works uniformly.

Key concept: Clarify the specific objectives and methods of large language model training.

Explanation: Considering that large language models mostly use autoregressive or autoencoding approaches, the training objective is often to maximize likelihood (maximize the probability of the training data), or equivalently to minimize prediction error. It is necessary to understand that during training, the model adjusts parameters to make predictions on training samples more accurate. Pay attention to differences in training objectives across models; for example, GPT and BERT have different training objectives. A common pitfall is ignoring the distinction between specific tasks.

Reference answer: The training objective of a large language model is usually to maximize the joint probability of word sequences in the training data, using the maximum likelihood estimation (MLE) method, by optimizing the model parameters so that the model can accurately predict each word in the sequence. The specific implementation depends on the model architecture; for example, the objective of an autoregressive model is word-by-word prediction, while an autoencoding model focuses on denoising or masked prediction.

Key concept: The interaction mechanism between the decoder and encoder and its role in sequence generation.

Explanation: Candidates should understand that the encoder is responsible for extracting contextual information from the input sequence and outputting representations (such as hidden states or encoding vectors), while the decoder uses this information combined with its own self-attention mechanism to gradually generate the output sequence. When answering, consider how the two work together through interaction (such as cross-attention), as well as the differences in the process between training and inference stages. Pitfalls include mistakenly thinking that the decoder works independently, ignoring the contextual information provided by the encoder, or misunderstanding the specific application of multi-head attention here.

Reference answer: Through interaction with the encoder, the decoder uses the context representation output by the encoder to guide the next-step generation of the already generated sequence. In each decoding layer, the decoder uses masked self-attention to focus on the content already generated, and at the same time, through the cross-attention mechanism with the encoder output, it uses the encoder's hidden states as keys and values, ensuring that the decoder can attend to the global information of the input. This cooperative mechanism enables the model to take into account both global semantic information and local context during generation, thereby achieving high-quality, semantically coherent sequence generation.

Key concept: Fine-tuning strategies and regularization techniques.

Explanation: Observe training and validation performance, use techniques such as data augmentation and early stopping, adjust the learning rate, choose an appropriate number of fine-tuned layers, and avoid updating all parameters, which can lead to overfitting. At the same time, combine regularization methods such as Dropout and weight decay to ensure the model's generalization ability on the new task.

Reference answer: By using a validation set to monitor performance, adopting data augmentation strategies, adjusting the learning rate, limiting the number of fine-tuned layers, and introducing regularization methods (such as Dropout) to reduce the risk of overfitting, the model's generalization ability can be effectively improved.

Key concept: Understand methods for preventing the model from losing its original knowledge when training on a new task.

Explanation: Consider using regularization techniques (such as ElasticWeightConsolidation), freezing some pretrained parameters, or adopting a multi-task learning strategy. In addition, monitor performance changes on the new task to ensure that the model does not over-adjust and cause performance degradation. Pay attention to adjusting the learning rate and the number of training epochs to avoid drastic changes to the pretrained weights.

Reference answer: By introducing regularization (such as EWC), freezing some pretrained parameters, using multi-task learning, or using a low learning rate and other strategies, the model can be ensured to retain its original knowledge while learning the new task, thereby effectively avoiding catastrophic forgetting.

Key concept: Understand the impact of data volume on fine-tuning results and the basis for judgment.

Explanation: It is necessary to consider the complexity of the task, the model capacity, and the target performance metrics. Generally speaking, the more data there is, the more rich features the model can learn, but it may also lead to overfitting. Whether the amount is sufficient should be judged through methods such as cross-validation, monitoring validation set performance, and avoiding data bias. At the same time, data augmentation or synthesis techniques can be used to supplement the insufficient part.

Reference answer: The amount of data should be reasonably selected according to task complexity and model capacity, combined with validation set performance and cross-validation results, to ensure that the data can both fully train the model and avoid overfitting. When data is insufficient, consider strategies such as data augmentation and transfer learning.

Key concept: Examine the balancing mechanism of decoding strategies in generative models between exploring new content and exploiting known information.

Explanation: Analyze how different decoding strategies (such as greedy search, beam search, sampling, etc.) affect the diversity and quality of model output. Consider that greedy search tends toward determinism and lacks diversity, sampling (such as temperature sampling and nucleus sampling) introduces randomness to increase exploration, and beam search trades off between search space and computational cost. Think about how to choose the appropriate strategy in different tasks, as well as potential pitfalls such as noise caused by excessive sampling or premature convergence caused by beam search. Ensure understanding of the applicable scenarios and limitations of each strategy.

Reference answer: Decoding strategies should be chosen according to the task objective; for example, when generating diverse content, use temperature sampling or nucleus sampling, while when pursuing high-probability output, greedy search or beam search can be used. Reasonably adjust parameters (such as temperature and beam width) to achieve a balance between exploration and exploitation, avoiding excessive exploration that degrades generation quality or excessive exploitation that leads to monotony. This balancing technique is key in the optimization of generative models.

Key concept: The relationship between the applicable scenarios of evaluation metrics and model performance.

Explanation: First, select the corresponding metric according to the task type (such as classification, regression, generation), then analyze the various characteristics of the metric, such as sensitivity, robustness, and preference for different types of errors. Consider the needs for accuracy, recall, or balance in practical applications, and avoid the bias brought by a single metric. At the same time, consider the performance of the metric under different data distributions to prevent misleading results. Pay attention to the differences between different metrics and avoid blindly relying on the limitations of a certain metric. Note that some metrics may be affected by factors such as class imbalance, so multi-metric comprehensive evaluation is needed.

Reference answer: Choosing an evaluation metric should be combined with the characteristics of the task and the performance requirements. Commonly used automatic evaluation metrics include accuracy, precision, recall, F1-score, as well as BLEU, ROUGE, METEOR, etc. in generation tasks. Appropriate metrics should be selected according to the specific task objectives, and multiple metrics should be used together for evaluation to comprehensively reflect model performance and avoid bias caused by relying on a single metric.

Key concept: Multi-metric evaluation strategies and the approach to avoiding bias.

Explanation: A single metric may cause the model to overfit to performance in one aspect and ignore the overall effect. A combination of multiple metrics should be used for evaluation, covering different aspects such as precision, recall, and semantic consistency. For example: evaluate simultaneously with accuracy, F1-score, and BLEU. Add human review or multi-task learning during model training to avoid being misled by machine metrics. When tuning parameters, also pay attention to the trend of metric changes, and avoid sacrificing other performance just to pursue a certain metric. Finally, consider the practical meaning of the metrics in combination with the business scenario to ensure that the overall model performance meets the requirements.

Reference answer: By adopting joint evaluation with multiple metrics, combining human evaluation, and considering business objectives, avoid the model over-optimizing a single automatic metric. Comprehensive evaluation from multiple perspectives can more fully reflect the model's actual application performance and ensure the model's robustness and practicality.

Key concept: Methods and processes for verifying the objectivity of human evaluation.

Explanation: To assess whether human evaluation of a model is biased or subjective, commonly used methods include having multiple evaluators score independently and then calculating consistency metrics (such as Cohen's Kappa), as well as designing reasonable scoring standards and conducting cross-validation to ensure the standardization and stability of the evaluation. The background differences of evaluators should also be considered; to reduce bias, training can be conducted or detailed evaluation guidelines can be formulated. It is necessary to guard against the risk of evaluator bias being introduced into the results and avoid evaluation results being dominated by a single subjective factor.

Reference answer: To verify the objectivity of human evaluation, multiple evaluators should be organized to score independently, and their consistency metrics (such as the Kappa coefficient) should be calculated to ensure that the evaluation has a certain degree of stability and consistency. Clear and detailed scoring standards should be established, and training and guidance should be provided to reduce subjective bias. If there are large differences, the causes of the bias should be analyzed, and the scoring process should be adjusted or evaluator training should be increased. At the same time, combine other objective metrics for comprehensive judgment to improve the reliability of the evaluation.

Key concept: Methods and strategies for reducing subjective bias in human evaluation.

Explanation: Identify the main sources of bias, adopt a multi-evaluator scheme, conduct multiple rounds of scoring, and calculate consistency. Formulate detailed scoring guidelines to ensure that each evaluator relies on unified standards. Provide training so that evaluators understand the evaluation standards and reduce differences in personal interpretation. Blind evaluation can be used, hiding information about the model source to avoid the evaluation being affected by preconceptions. Statistically analyze the evaluation results, analyze bias and consistency, and adjust the scheme. The ultimate goal is to reduce the impact of personal bias through standardization and diversity.

Reference answer: By training evaluators, establishing unified and detailed scoring standards, and using cross-scoring by multiple evaluators, subjective bias can be significantly reduced. Blind evaluation (hiding model information) is also an effective means to reduce preconceived bias. In addition, statistical consistency metrics can help identify bias and make adjustments when necessary. A systematic process and diverse evaluation help maintain the fairness and objectivity of the evaluation.

Key concept: The principles and implementation techniques of model quantization.

Explanation: Consider different quantization schemes (such as fixed-point quantization, dynamic range quantization, binarization, quaternarization) and their impact on model accuracy and computational efficiency. Analyze the factors that need to be considered in the quantization process: numerical range, quantization error, hardware support. Be careful not to ignore the accuracy loss caused by quantization and its correction strategies.

Reference answer: Model quantization greatly reduces storage space and accelerates inference by converting high-precision floating-point parameters into low-bit-width integers (such as 8-bit, 4-bit, binary, etc.). Under the premise of ensuring appropriate accuracy, this can reduce model complexity and improve efficiency, and is especially suitable for deployment on edge devices.

Key concept: Understanding the model's ability to maintain and use contextual information in a conversation.

Explanation: Analyze how the model captures long-distance dependencies through contextual information in the input sequence, focusing on the attention mechanism and memory ability between tokens. Consider the model's input length limit, the design of the attention mechanism, and the way dialogue history is managed. Be careful to avoid focusing only on a single sentence and ignoring the continuity of the context; also consider the risk that the model may truncate information because the context is too long.

Reference answer: Large language models capture contextual information in the input sequence through the self-attention mechanism and can understand the related content before and after in a conversation. With the help of a pretrained encoder, the model can remember longer dialogue history and enhance context understanding ability. In practical applications, dialogue history concatenation or specially designed context management strategies are often used to ensure that the model maintains coherence in multi-turn dialogues.

Key concept: Methods for enhancing model memory and coherence.

Explanation: Consider introducing long short-term memory mechanisms, cascaded models, or multi-turn information passing strategies to enhance context understanding. Evaluate the length management of dialogue history to avoid information loss. At the same time, a caching mechanism can be combined to fix important information from previous text, or specialized dialogue state tracking technology can be used. Pay attention to the balance between the model's capacity limits and information selection, and avoid performance degradation caused by information overload.

Reference answer: Methods to improve the continuation ability of a dialogue system include adopting memory enhancement mechanisms (such as long-term memory modules), optimizing context management strategies, and introducing dialogue state tracking technology. These measures can help the model better maintain dialogue coherence, reduce information loss, and ensure content consistency and relevance in multi-turn dialogues. Combined with task-oriented design, enhance the relevance and deep understanding ability of the model's dialogue.

Key concept: Privacy risk assessment methods and strategies.

Explanation: Conduct a comprehensive review from data collection, storage, and processing to model output, and identify potential privacy leakage paths. Consider attack models in the scenario, such as reverse inference and reverse application of the model. Use privacy impact assessment (PIA) tools to systematically analyze possible risks, and simulate attack scenarios to test the effectiveness of privacy protection measures. While ensuring compliance, formulate response strategies to reduce risks.

Reference answer: By comprehensively reviewing the data flow, combining threat modeling and privacy impact assessment, identify potential leakage paths for sensitive information, verify the effectiveness of existing protection measures, and formulate response plans. Continuous monitoring and improvement of privacy protection measures is key.

Key concept: Multimodal fusion techniques and their improvement effects.

Explanation: Multimodal filtering combines different information sources (such as text, images, video, and audio) to jointly determine whether content is harmful. Fusion methods include feature-level fusion and decision-level fusion, which improve the comprehensiveness and accuracy of detection. It is necessary to consider feature extraction methods for different modalities, data alignment, and model integration issues. Multimodal techniques can compensate for the shortcomings of a single modality and are better at identifying complex harmful content.

Reference answer: Multimodal filtering improves detection accuracy and robustness by integrating information from multiple data sources, and it performs better especially when facing complex or subtle harmful content.

Key concept: Security policies and practices for cloud deployment.

Explanation: Consider the security of data transmission (such as encrypted transmission protocols), storage security (permission control, encrypted storage), access control (authentication and authorization mechanisms), and the security of API interfaces (preventing unauthorized access and attacks). Also pay attention to dependency security and environment isolation to avoid privilege escalation operations or information leakage. An error-prone point is neglecting permission management or failing to patch in time, leading to potential vulnerabilities.

Reference answer: Adopt TLS/SSL encrypted communication, use strict identity authentication and permission control, ensure storage security (such as encrypted storage and access auditing), apply network security policies (such as VPC and firewalls), and at the same time integrate security policies into continuous monitoring and vulnerability patching processes.

Key concept: Selection and application of monitoring metrics and tools.

Explanation: It is necessary to set up monitoring of key metrics (such as response time, throughput, error rate, system resource usage, etc.), combined with APM tools (such as Prometheus and Grafana) or monitoring platforms provided by cloud vendors. An alerting mechanism should also be configured to promptly notify maintenance personnel of abnormal metrics. Pay attention to the correlation between log analysis and metrics to quickly locate the root cause of problems. An error-prone point is incomplete monitoring metrics or failure to preset thresholds, resulting in problems not being discovered in time.

Reference answer: Collect performance metrics and system resource data, use visualization tools for real-time monitoring, set reasonable warning thresholds and automated alerts, and combine log analysis for troubleshooting, ensuring that the operating status of the model deployment remains continuously stable.