Skip to content
mentorship

AI-native engineering work sample

Agentic ML codebase lab

The model can write code. Your signal is whether you can map the system, delegate bounded work, reject bad changes, and prove the result.

60 minutesExisting multi-file Python package, failing tests, one bug, one feature

Do this only if the format matches

Confirm the round with recruiting first. This is an original practice exercise, not a claim that a specific company uses this exact prompt.

Protocol

  1. Run the tests and write a five-line codebase map before opening the agent.
  2. State the merge invariant and slice-report contract in your own words.
  3. Delegate one bounded behavior at a time. Include the file, constraint, and verification command.
  4. Inspect every diff. Revert unrelated edits and explain one rejected suggestion.
  5. Add one edge test the agent did not propose, then run focused and full verification.
  6. End with a two-minute summary of the change, residual risk, and what you would inspect next.

Deliverables

  • A repaired merge operation and implemented slice report.
  • One candidate-written regression test.
  • A short prompt log showing bounded delegation.
  • A final diff you can explain line by line.

Graduation gates

01

Merged shard metrics equal single-pass metrics.

02

Low-support slices cannot trip the guardrail.

03

The public API remains stable.

04

No generated change survives without review and a verification path.

Starter files

Answer key is a rubric, not a solution dump

The linked question explains what strong performance looks like, common failure modes, and senior follow-ups. Attempt the lab before reading it.

Read the interview rubric →

← All executable labs