What needs to be finished first
A list of the future consumers of the Ontology, Knowledge Graph, or GraphRAG pipeline is available so questions come from a real usage context.
What needs to be ready before start
Shared document or table with columns question, example answer, required concepts, priority, coverage status; sample questions as anchors; whiteboard for clustering.
A facilitator with ontology or modeling experience; two to four domain experts; one or two consumers (for example a data scientist or GraphRAG developer); a scribe for the question table.
Use-case description of the planned model; known data sources; existing vocabularies or ontologies for reuse; sample questions from similar projects as anchors.
2-4 h
Prepare the table. Paste 3-5 sample questions as anchors (for example 'Which components depend on service X?'). State the rule: questions must be answerable by the model, not by Wikipedia.
The one question this method answers
Which concrete domain questions must the planned model be able to answer so that it is sufficient for the planned use cases and not oversized?
Marker: Phase
| Step | Duration | Action | Hint |
|---|---|---|---|
1Phase 1: Use-case briefing | 20-30 min | Consumers describe their use cases concretely. One to two example questions per use case. Clarify what Wikipedia or search can already deliver and where the model must add real value. | If use cases stay vague ('somehow AI on top of it'), park them for now. Without a real question there is no useful competency question. |
2Phase 2: Gather questions | 45-60 min | Silent phase plus round robin: each person formulates concrete domain questions the model must answer. At least 30 questions. One question per row. Include a concrete trigger example. | If a question starts with 'What is ...?', it is usually a definition question rather than a modeling question. Rephrase it to 'Which ...?', 'How many ...?', or 'Which relationship ...?'. |
3Phase 3: Cluster and prioritize | 30-45 min | Cluster the questions (for example entity lookup, relationship query, aggregate, temporal question). Discuss coverage per cluster. Set priority: must-have, nice-to-have, out-of-scope. Mark out-of-scope explicitly. | Out-of-scope questions are valuable as boundary definition. Removing them instead of marking them means losing the reason for later scope discussions. |
4Phase 4: Derive concepts and test queries | 30-45 min | For each must-have question, list the concepts, relationships, and properties the model needs. Sketch test queries from the top 3-5 questions as pseudocode (SPARQL, Cypher, GraphQL). | If the same concepts show up in many questions, those are the core classes. If a question needs concepts that appear nowhere else, check whether it is truly must-have. |
What comes out at the end
Table or Markdown document with numbered questions, priority, derived concepts, example test query, and coverage matrix against use cases. Serves as the acceptance basis for the model.
Date and model version in the header. Let the question set grow in parallel with the model. Mark answered questions with a link to the model element instead of deleting them. Keep out-of-scope questions with a reason.
Compact working template for Competency Questions with context, input, output artifacts, and next step.
| Element | Description | Rating | Evidence | Owner | Next step |
|---|---|---|---|---|---|
| 1 | |||||
| 2 | |||||
| 3 |
What consequence follows from the matrix?
Concrete filled scenario, fictional example
Concrete filled scenario, fictional example
Use Case 1: Incident Response: quickly see who is affected by an outage. Use Case 2: Architecture Review: find bottlenecks and single points of failure.
MATCH (s:Service {name:'PaymentService'})<-[:DEPENDS_ON]-(c:Service)
RETURN c.name
Coverage v0.3: 9 of 12 must-have questions answerable. Gaps: questions 7, 10, 11 (concept SecretStore missing).
Recognize symptoms and steer against them
Modelers invent questions that nobody will ask, and test queries are not used in practice.
Bring at least two real consumers into the workshop. If that is not possible, run short interviews first and bring questions from those interviews.
Questions like 'What is a microservice architecture?' appear in the list.
Such questions are glossary work, not model work. Rephrase into 'Which services are marked as microservice-style?' or remove them. Keep the glossary in a separate file.
More than 80% of the questions are marked must-have, and prioritization is based on gut feeling.
Use a strict definition: must-have means it blocks a concrete use case. When in doubt, move it to nice-to-have. Use 60% must-have as a rough upper bound.
Questions are written down, but nobody can explain how they would look as a technical query.
Sketch at least 3 top questions as pseudocode queries. If that is not possible, the question is either unclear or the model paradigm is wrong.
The question set is written once and then never updated. Answered questions are not linked.
Keep the question set next to the model in the repo. Check coverage per model iteration and maintain links. A CI check can make gaps visible.
Done signals checkable in under a minute
Go to the profile for purpose, similar methods, and sources or continue to the next method in the catalog.