Semantic Graph Memory Inspired by Neuroscience

Introduction

Large Language Models (LLMs) are incredibly powerful, yet they face a major limitation: a fixed context window that restricts how much information they can "remember." This forces developers to carefully choose what to include in each prompt, often resulting in lost context and shallow knowledge integration. Traditional retrieval-augmented generation (RAG) methods inject text snippets into prompts, but simple vector searches can fall short when nuanced context or multi-hop reasoning is needed. These shortcomings can lead to hallucinations and unstable long-range coherence in AI outputs.

To overcome these barriers, we need a semantic memory structure --a system that can persistently store and recall facts and their interrelationships. Inspired by the way human memory works, knowledge graphs offer a compelling solution. By representing information as interconnected nodes and edges, a semantic graph provides a rich, structured memory that an LLM can query not only for facts but also to follow transparent reasoning chains.

Model Context Protocol

I recently built such a semantic graph memory system using the Model Context Protocol (MCP). The idea began with the insight that persistent memory should support not just retrieval, but also logical reasoning. Although we model artificial neural networks on brain-inspired architectures, many of the brain's methods for learning, memory, and reasoning have yet to be integrated into our AI systems.

In my implementation, I defined a schema with seven node types:

1. Entity: People, organizations, products, and physical objects.

2. Event: Time-bound occurrences with temporal details.

3. Concept: Abstract ideas, theories, principles, and frameworks.

4. ScientificInsight: Research findings supported by evidence.

5. Law: Established principles, rules, or regularities.

6. ReasoningChain: Structured representations of logical reasoning.

7. ReasoningStep: Individual steps in a reasoning process.

Additionally, as content is processed, key emotional and cognitive features are captured:

• Emotional valence and arousal ratings
• Abstraction levels for concepts
• Evidence strength for scientific insights
• Causal relationships between events
• Confidence scores for reasoning chains and relations

While this schema works well for general purposes, additional node types can be added for specialized applications.

When users interact with a client application (for example, Claude Desktop), the system retrieves relevant nodes and their relationships from the graph database to form the context of the conversation. As users converse, upload documents, or search the web, new content is continuously processed and added to the graph.

Reasoning Chains

A standout feature of this system is its ability to create and maintain reasoning chains. When an LLM tackles a complex query--perhaps by breaking it down into multiple steps--the MCP server captures each step as a separate node in the graph. For example, if the model is asked a complex question, it might:

1. Retrieve a fact about X.
2. Combine this with knowledge Y to infer Z.
3. Draw a final conclusion.

Each intermediate step is recorded as a ReasoningStep node and linked into a ReasoningChain. This chain is then returned as part of the answer, providing a clear rationale that can be revisited or adapted in future similar queries.

Code

https://github.com/shuruheel/mcp-neo4j-shan/tree/main

Next Steps

While the project is promising, there are areas for improvement:

Batch Processing: Developing a tool to process and ingest an existing knowledge base.

Tool Calling: Refining the mechanism so the system reliably calls the correct tool.

Advanced Graph Traversal: Implementing more complex algorithms to better retrieve the most relevant nodes for each context.

By enhancing these areas, we can further improve the performance and reliability of semantic graph memory systems, paving the way for even more robust applications in AI.