The description is a selection boundary
A skill can contain excellent instructions and still fail to help if the agent selects it for the wrong work. Trigger evaluation asks a narrower question than output evaluation: given a request, does the agent decide to use this skill?
OpenCode Skill Creator provides skill_eval for this part of the workflow. The value comes from the cases you give it, not from the existence of a score. A test set made only of obvious matches can hide a description that activates far too broadly.
Write both sides of the boundary
For a Docker Compose review skill, a positive case could ask for a review of a provided Compose configuration. A negative case might ask for a definition of a container image, or request changes to a Kubernetes manifest. The topics are related, but the intended tasks differ.
Add near misses, too. A request that mentions Docker while asking for application code should not automatically activate a Compose review skill. Conversely, a request can be a valid match without repeating the skill’s exact name.
Should trigger:
Review this Docker Compose file for risky defaults.
Check the service definitions in my compose.yaml.
Should not trigger:
Explain the difference between an image and a container.
Review this Kubernetes Deployment manifest.These are planning examples, not a copy-pasteable tool input schema. The repository’s Docker Compose helper example includes an eval set in the tool’s actual JSON format.
Separate training from the decision
Keep some examples out of the improvement prompt. Training cases help identify wording problems; held-out cases help you judge whether the revised description works beyond those examples.
Do not move a difficult held-out case into the training set simply to raise the reported result. If you intentionally change the test set, record that change and keep the old results separate. Otherwise, you are comparing different questions.
Read false positives and false negatives
A false positive means a negative case activated the skill. Look for a description that describes a broad topic rather than a specific job. A false negative means a positive case did not activate it. Look for missing user language, unclear context, or an overly narrow description.
Use those patterns to choose a focused edit. Do not stuff the description with every phrase from the test set. The goal is a clearer boundary, not a transcript of the examples.
Evaluate this skill with labeled positive, negative, and near-miss cases using skill_eval. Keep the held-out set separate. Explain the false positives and false negatives before suggesting a description change.Preserve each item’s identity
The plugin tracks evaluation results per eval-set item rather than by query text. Two entries with identical text can remain distinct items with their own labels and counts. Preserve item order and identity when you organize or compare results; do not collapse results into a map keyed only by the query string.
Keep the score in context
Model behavior can vary across repeated runs, and evaluation uses your configured provider. Consider that variability and the cost of a larger run before expanding the set. A useful report includes the cases, the candidate description, the run conditions, and the failure patterns.
Finally, review the actual task output. Trigger evaluation tells you about selection. It does not prove that the selected skill produces correct configuration advice or a better result for the user.