Xcode 27

When code can't judge quality: model-as-judge evaluators in Swift

When Code Can't Judge Quality: Model-as-Judge Evaluators in Apple's Evaluations Framework

Code-based evaluators can only measure what code can measure. This companion article goes deep on ModelJudgeEvaluator: choosing a scoring scale, writing scoring levels that actually work, and wiring a model-as-judge into the TripPlanner evaluation from part one.

Apple's Evaluations framework: testing non-deterministic Foundation Models features

Testing the Untestable: A Deep Dive into Apple's Evaluations Framework

How to test generative, non-deterministic features with Apple’s Evaluations framework, the five steps of an evaluation, datasets and the Loader protocol, synthetic sample generation, evaluators and metrics, and the compiler gotchas hit along the way.