Building a Company GPT: SME Guide 2026 with RAG and Workflow Automation

How small and medium-sized enterprises build their own Company GPT. With RAG system, n8n workflows, and GDPR-compliant infrastructure for your own AI assistant.

Author: SIMO GmbH
Published on: June 18, 2025
Reading time: 8 minutes
Category: AI Tools & Technology

A Company GPT of your own is no longer a dream of the future but a concretely achievable solution for small and medium-sized enterprises. What this means is an AI system based on your own company knowledge that answers employee questions and can be integrated into existing business processes—without sensitive data having to flow to external providers.

This guide shows step by step how SMEs build their own AI assistant, which technologies are needed, and what to consider regarding data protection and governance.

What a Company GPT Means for SMEs

A Company GPT is essentially a language model linked to internal company knowledge. Instead of drawing on general internet knowledge, it responds based on your documents, process descriptions, price lists, and customer information.

The advantages are obvious:

Instant Answers: Employees ask questions in natural language and receive precise answers with source citations.
Knowledge Preservation: The expertise of experienced employees is systematically captured and made accessible.
Faster Onboarding: New team members find their way more quickly because they have access to an intelligent knowledge database.
Process Efficiency: Recurring questions no longer consume human work time.

The Technical Architecture: RAG as the Foundation

The heart of a Company GPT is a RAG system (Retrieval Augmented Generation). RAG combines two strengths: the ability of a language model to understand and generate natural language, and the precision of a vector database that finds relevant documents.

How RAG Works

Ingestion: Company documents are read in, divided into sections (chunks), and stored as mathematical representations (embeddings) in a vector database.
Retrieval: When a user asks a question, it is also converted into an embedding. The vector database finds the most semantically similar text sections.
Generation: The found text sections are passed to the language model together with the question, which formulates a precise answer from them.

The Technology Stack

The following stack has proven effective for SMEs:

Component · Recommended Solution · Task
Workflow Engine · n8n · Orchestration of all processes
Vector Database · Supabase with pgvector · Storage of embeddings
Embedding Model · OpenAI text-embedding-3-small · Converting text to vectors
Language Model · GPT-4, Claude, or Llama · Answer generation
Re-Ranker · Cohere Rerank · Quality optimization of search

This stack is cost-efficient, scalable, and does not require its own IT department.

Step-by-Step Setup

Step 1: Identify Data Sources

Before you begin implementation, take inventory of your company knowledge:

Structured Documents: SOPs, manuals, price lists, contracts
Unstructured Documents: Emails, meeting notes, wiki entries
Systems: CRM data, ERP information, ticketing systems

Start with the 20 to 50 most important documents. A local folder or a Google Drive is perfectly sufficient as a starting point.

Step 2: Document Preparation

The quality of your Company GPT stands and falls with the quality of prepared documents:

PDF Extraction: PDFs with a text layer are unproblematic. Scanned documents require OCR.
Cleaning: Remove headers, footers, signatures, and duplicate content.
Metadata: Each document should be tagged with mandatory fields: department, document type, validity period, source.
Chunking: Split texts into meaningful sections of 512 to 800 tokens, with 10 to 20 percent overlap.

Step 3: Build the Ingestion Workflow in n8n

The first n8n workflow connects your data sources to the vector database:

Trigger: Scheduled (daily) or event-based (on file update)
Read data source: Google Drive, SharePoint, or local folder
Process documents: Extraction, cleaning, chunking
Generate embeddings: Call OpenAI API
Store in Supabase: Chunks with vectors and metadata

Step 4: Create the Q&A Workflow

The second workflow processes user queries:

User asks a question (via chat interface, Slack, or email)
Question is converted into an embedding
Vector search in Supabase with metadata filters
Re-ranking of results with Cohere
Language model generates an answer based on the top matches
Answer with source citations is returned

Step 5: Access Rights and Governance

Row Level Security (RLS) in Supabase ensures that users can only access authorized documents.
Restrict the system prompt: The model may only respond based on the delivered documents. When information is missing, it must communicate this transparently.
Capture audit logging: Who asked which question when? Plan for this from the start in compliance-sensitive industries.

Practical Examples from the Mid-Market

Skilled Trades Business with 18 Employees

A sheet metal shop fed its Company GPT with maintenance instructions, supplier terms, and measurement forms. New employees now get their answers in seconds instead of minutes. The system has been running stably for months without notable maintenance.

Timber Construction Company with 22 Employees

A timber construction firm uses its Company GPT for automatic classification of incoming inquiries, proposal preparation, and as an internal knowledge base for construction plans and supplier lists.

B2B Service Provider with 25 Employees

A service company uses its Company GPT for lead qualification: The system analyzes lead data from CRM and website, creates automatic scorings, and prioritizes inquiries. The conversion rate rose from 8 to 12 percent.

Costs and ROI

Ongoing Costs

For a typical SME setup with OpenAI embeddings, Supabase Pro, and n8n Cloud, the monthly costs are in the low double-digit euro range. Costs scale with document volume and the number of queries.

One-Time Costs

Document preparation: The largest initial effort. Depending on document volume, 1 to 5 working days.
Workflow setup: 1 to 2 days for the prototype, 2 to 4 weeks for a production-ready system.
Training: AI literacy training for employees (a legal requirement since February 2025).

Typical ROI

Use Case · Typical Annual Savings · Typical Year 1 ROI
Internal knowledge base · 20,000 - 40,000 EUR · 200 - 400 %
Automatic inquiry classification · 30,000 - 50,000 EUR · 300 - 500 %
Lead qualification · 50,000 - 180,000 EUR · 400 - 650 %
AI customer service · 40,000 - 60,000 EUR · 100 - 200 %

GDPR and EU AI Act

GDPR Requirements

Data Processing Agreements (DPA): Conclude with all AI providers (OpenAI offers a DPA).
Data Minimization: No personal data in the knowledge base unless absolutely necessary.
Access Control: RLS in Supabase, explicit filters in all queries.

EU AI Act

AI Literacy Obligation: In effect since February 2025. All employees who use AI tools must be demonstrably trained.
Transparency Obligation: Fully applicable from August 2026. Document which data flows through the system.
Risk Classification: An internal knowledge system is typically not classified as a high-risk system but still requires documentation.

The 90-Day Plan for Your Own Company GPT

Phase 1 (Day 1-30): Foundation

Conduct process and data audit
Identify and prepare 20-50 core documents
Set up n8n and Supabase
Define metadata standards
Conduct AI literacy training

Phase 2 (Day 31-60): Prototype

Build ingestion workflow and index first documents
Create Q&A workflow with simple chat interface
Create a Golden Set with 10-30 test questions and evaluate
Configure access rights

Phase 3 (Day 61-90): Production

Connect system to additional data sources
Integrate re-ranking
Activate audit logging
Roll out to all relevant employees
Start ROI measurement

Frequently Asked Questions

Do I need an IT department for a Company GPT?

No. With n8n as a workflow tool and Supabase as a managed database, a functional system can be built without programming skills. The tools are designed so that technically proficient business users can operate them.

How do I keep my Company GPT up to date?

The ingestion workflow can run on a schedule (daily, weekly) or event-driven (on file updates). This way, the knowledge base automatically stays current.

What happens if the system gives a wrong answer?

A correctly configured system prompt forces the model to respond only based on the delivered documents. When information is missing, it responds with “I do not have verified information on that.” Additionally, a Golden Set evaluation process ensures systematic quality assurance.

How do I protect confidential information?

Row Level Security in Supabase, metadata-based filtering, and clear access rights ensure that each user only sees the documents intended for them.