When I first started thinking about how teams of humans and AI agents could work together on complex analytical problems, I kept coming back to one fundamental challenge: How do we create a structure that preserves the rigor of scientific methodology while accommodating the fluid, iterative nature of human-AI collaboration?

Part of this thinking came from a personal aspiration: seeing that AI might replace a lot of the work I am currently doing, I realized it might be more valuable to focus deeply on working with AI itself, leveraging its capabilities to do the heavy lifting. I wanted to develop a system capable of producing meaningful analysis – such as trading insights – that could generate income autonomously, without the constant need for client interactions. This idea fuelled our initial focus on trading analysis and later expanded into a broader framework.

Complex analytical challenges increasingly require both human expertise and computational capabilities. With the emergence of agentic LLM-based systems as a key novel player, these frameworks must now also account for AI’s ability to autonomously generate insights and adapt in real time. Whether in policy evaluation, scientific research, or market analysis, we need structures that effectively integrate this hybrid approach to problem-solving.

The solution we’ve developed represents analytical progress as a directed graph, where each node captures a hypothesis, observation, or conclusion, and edges represent the logical relationships between these elements. But unlike traditional graph databases, our framework is specifically designed for incremental, collaborative analysis.

As we built this system, it became evident that many of these insights were already present, waiting to be unlocked. What changed was the advent of AI systems that brought these insights into focus, transforming them into actionable outcomes. For the first time in history, we can interact with intelligence that is available on demand, 24/7, and without the typical friction found in “real-world” collaboration. This capability has not only reinforced what we suspected but also opened doors to possibilities I hadn’t initially anticipated.

One of the enabling aspects of our framework is its integration with GitHub’s infrastructure. This integration enables truly asynchronous collaboration between humans and AI agents through familiar collaboration workflows:

– Issues can be created by both humans and AI agents to propose new hypotheses or analysis paths

– Comments on issues facilitate detailed discussions about analytical approaches

– Issue labels track the state of different analysis components

– Notifications keep all participants informed of updates and required actions

– Direct updates to the graph are made after consensus through issue discussions

This infrastructure leverages GitHub’s proven collaboration tools while maintaining our graph-based analytical rigor. For example, when an AI agent identifies a potential pattern, it can:

1. Create an issue with the initial hypothesis

2. Tag relevant team members for input

3. Provide initial analysis in comments

4. Wait for discussion and consensus

5. Update the graph once validation criteria are met

Similarly, human analysts can:

1. Create issues for questions they want AI agents to explore

2. Comment on AI-generated analyses

3. Suggest modifications to analytical approaches

4. Validate and approve graph updates through issue discussions

This asynchronous workflow ensures that neither humans nor AI agents become bottlenecks in the analytical process, while maintaining clear documentation of all decisions and interactions through GitHub’s issue system.

1. Human-AI Synergy

What fascinates me about this approach is how it naturally accommodates different types of intelligence. Humans excel at providing context and making nuanced judgments, while AI agents can process vast amounts of data and identify patterns. The graph structure allows each to contribute to their area of strength while maintaining analytical rigor.

This synergy also aligns with a workshop I developed, which teaches people to think in “prompt chains” – a method for reasoning about human-AI collaboration that helps unlock new ways to approach analytical challenges.

Consider this example from our research:

“`json

{

“metadata”: {

“objective”: “Identify high-probability retracement entry”,

“context”: “Market Analysis”,

“team”: “Mixed Human-AI Research Group”

}

}

“`

This metadata structure ensures that every analysis has a clear purpose and context, regardless of whether the next step comes from a human researcher or an AI agent.

1. Incremental Knowledge Construction

One of the most powerful aspects of this framework is its support for incremental progress. Each node in the graph represents a validated piece of analysis that becomes a building block for further investigation. The system enforces what I call “constructive validation” – new contributions must build coherently on validated previous work.

For instance, when a new hypothesis is proposed:

“`python

def validate_hypothesis(self, hypothesis):

required_fields = [

“goal_relevance”,

“data_source”,

“validation_criteria”,

“probability”

]

“`

This validation ensures that each new piece of analysis contributes meaningfully to the overall objective while maintaining scientific rigor.

1. Transparent Accountability

Perhaps the most crucial aspect for research and policy work is the system’s inherent transparency. Every analytical step, whether taken by human or AI, is logged with clear attribution and validation criteria. This creates what I like to call a “living audit trail” – not just a record of what was done, but how and why decisions were made.

Sharing these ideas in a blog post felt natural, as it’s a way to bring others into the evolving conversation about how we can bridge human creativity and AI’s analytical power.

While our current implementation demonstrates the framework’s capabilities in market analysis, the underlying principles suggest promising applications across various analytical domains:

– Policy Impact Assessment: The framework could enable teams to build up evidence and analysis incrementally, maintaining clear traceability from initial assumptions to final policy recommendations

– Scientific Research: The structured collaboration model could support multiple researchers and AI tools working together on complex problems, with built-in validation of each contribution

– Complex Decision Making: Organizations could leverage the framework to maintain rigorous analytical standards while combining human expertise with AI capabilities

These potential applications share common requirements that our framework addresses: the need for incremental progress, clear validation criteria, and transparent collaboration between human and artificial intelligence.

What excites me most about this framework is its potential for scaling complex analytical work without sacrificing rigor or transparency. Our initial implementation in market analysis serves as a proof of concept for several key principles that could be valuable across different domains:

1. Enhanced Validation Mechanisms: The framework’s current validation system demonstrates how we can maintain analytical rigor in collaborative environments. This could be expanded to address domain-specific validation requirements in policy analysis or scientific research.

2. Structured Collaboration: Our implementation shows how human expertise and AI capabilities can work together within a formal framework. This structured approach to collaboration could be particularly valuable for research teams working on complex, multi-faceted problems.

3. Transparent Reasoning: The graph-based structure provides clear documentation of analytical steps and decision-making processes – a critical requirement for both policy work and scientific research.

The future of complex analysis lies not in choosing between human expertise and artificial intelligence, but in creating frameworks that allow both to contribute optimally. This graph-based approach, integrated with GitHub’s collaborative infrastructure, represents a step toward that future – one where rigorous, incremental analysis can scale to meet increasingly complex challenges while maintaining the highest standards of scientific integrity.

The example we’ve explored, while rooted in market analysis, demonstrates principles that could be universal to any complex analytical task: the need for clear validation criteria, transparent reasoning, and structured collaboration between human and artificial intelligence. The addition of GitHub’s asynchronous collaboration tools provides a practical infrastructure for implementing these principles in real-world scenarios.

What makes this framework particularly promising is how it transforms the challenge of human-AI collaboration from a theoretical concept into a practical methodology, supported by proven development tools and workflows. It suggests a concrete way forward for researchers and policy makers looking to leverage the best of both human and artificial intelligence in their analytical work, while maintaining the rigor and transparency needed for serious research and policy development.

Let me know if you need any additional details! Cheers, Rene Luijk

LinkedIn: https://www.linkedin.com/in/reneluijk/

X: https://x.com/Rene_Luijk

Also, you can join one of the workshops that offer a unique opportunity to discover how different computing tools can work together as a team.

https://www.synestheticminds.com/en/workshop

Oryginally published at:

https://spektrumlab.io/human-ai-collaboration-a-graph-based-framework-for-analysis-by-rene-luijk/