Machine learning and AI engineer interview questions
ML and AI interviews test whether you understand models well enough to choose, evaluate and debug them, not just call a library. Many now also cover large language models: prompting, retrieval-augmented generation (RAG) and evaluating outputs.
The questions below are grouped by level, with what a strong answer should include.
What interviewers assess
- ML fundamentals: bias and variance, overfitting, features and common algorithms
- Evaluation: choosing metrics, validation and data leakage
- Deep learning and LLMs: training basics, embeddings, RAG and evaluation
- Production: deploying, monitoring and improving models with real data
Basic questions
Explain the bias-variance trade-off.
What a good answer covers: High bias underfits, high variance overfits; more data, regularisation or simpler models reduce variance; more features or capacity reduce bias. Tie it to learning curves.
What is the difference between supervised and unsupervised learning?
What a good answer covers: Labelled targets (classification, regression) versus finding structure without labels (clustering, dimensionality reduction), with a real example of each.
How do you prevent overfitting?
What a good answer covers: Cross-validation, regularisation, early stopping, dropout, more or augmented data, and simpler models.
Why is accuracy a bad metric for imbalanced data?
What a good answer covers: A model predicting only the majority class can look accurate. Use precision, recall, F1, PR-AUC, or a cost-based metric that matches the business problem.
How do you split data into training, validation and test sets?
What a good answer covers: The test set is kept untouched until the end; validation is for tuning; use time-based splits for time series and grouped splits to avoid leakage.
Intermediate questions
What is data leakage? Give an example.
What a good answer covers: Information from the future or from the target sneaks into training, such as scaling with statistics from the whole dataset or a feature recorded after the outcome. Explains suspiciously good results.
Compare random forests and gradient boosting.
What a good answer covers: Bagging of independent trees versus sequential trees that correct errors; boosting is often more accurate but needs more tuning and is easier to overfit.
What are embeddings and how are they used?
What a good answer covers: Dense vectors that place similar items close together; used for search, recommendations, clustering and RAG retrieval.
Explain how a retrieval-augmented generation (RAG) system works.
What a good answer covers: Chunk and embed documents, retrieve the most relevant chunks for a query, add them to the prompt and generate a grounded answer. Mention chunking, re-ranking and citing sources.
How would you choose a threshold for a classifier?
What a good answer covers: Based on the cost of false positives versus false negatives, using precision-recall curves on validation data, agreed with the business.
Advanced questions
How do you evaluate an LLM-based feature before release?
What a good answer covers: A test set of realistic inputs, clear criteria, automated checks plus human review, model-graded evaluation used carefully, and tracking regressions as prompts or models change.
Your model performs well offline but poorly in production. Why might that be?
What a good answer covers: Training-serving skew, data drift, leakage in the offline evaluation, a different user population, or feedback loops. Explain how you would check each.
How do you monitor a model in production?
What a good answer covers: Input data drift, prediction distribution, latency and errors, and real outcome metrics when labels arrive, with alerts and a retraining plan.
Explain the transformer attention mechanism at a high level.
What a good answer covers: Each token builds a weighted mix of other tokens' representations using query, key and value projections, which lets the model use context across the whole sequence in parallel.
When would you fine-tune a model rather than use prompting and RAG?
What a good answer covers: Fine-tune for consistent style, format or a narrow task with plenty of examples; use RAG for knowledge that changes or must be cited; prompting first because it is cheapest. Discuss cost and maintenance.
Common mistakes
- Reporting accuracy only, with no baseline and no discussion of the right metric.
- Describing a Kaggle notebook as production experience.
- Not being able to explain why your model works, only which library you used.
- Ignoring data quality, which is where most real projects spend their time.
How to prepare
- Pick one project and be ready to defend every choice: data, features, model, metric and results.
- Revise evaluation metrics and when each one misleads.
- Build a small RAG or LLM project and think about how you would evaluate it.
- Practise explaining one algorithm simply, as if to a product manager.
Reading questions only goes so far: the real test is answering them aloud, under follow-up questions. Career Scan builds a mock interview from your own resume and the job, then shows exactly which answers need work.
Frequently asked questions
Do ML interviews include coding?
Yes, usually Python and data manipulation, sometimes implementing a simple algorithm, plus general coding for engineering roles.
How much maths is needed?
Enough to explain gradients, probability, linear algebra basics and how loss functions work. Research roles go deeper.
Are LLM questions common now?
Increasingly. Many AI engineer roles ask about prompting, RAG, evaluation and the cost and latency of using large models.