Optimisez votre base de connaissances avec l'IA et Milvus

Ce workflow n8n vous permet de scrapper automatiquement les derniers essais de Paul Graham, d'extraire le contenu textuel et de le charger dans une base de données vectorielle Milvus. Grâce à l'intégration d'OpenAI, les textes sont enrichis par des embeddings puissants qui facilitent leur recherche et analyse. Utilisez cet outil pour booster la gestion de votre base de connaissances avec des capacités avancées d'intelligence artificielle, tout en économisant un temps précieux grâce à l'automatisation.

85,773 vues
20,785 copies
Data

Documentation Complète

📋 Optimisez votre base de connaissances avec l'IA et Milvus

💡 Description

Ce workflow n8n vous permet de scrapper automatiquement les derniers essais de Paul Graham, d'extraire le contenu textuel et de le charger dans une base de données vectorielle Milvus. Grâce à l'intégration d'OpenAI, les textes sont enrichis par des embeddings puissants qui facilitent leur recherche et analyse. Utilisez cet outil pour booster la gestion de votre base de connaissances avec des capacités avancées d'intelligence artificielle, tout en économisant un temps précieux grâce à l'automatisation.

📈 Impact & ROI: Réduisez le temps passé sur la collecte et l'organisation des informations tout en augmentant la précision et l'accessibilité des données grâce à l'intelligence artificielle.

🚀 Fonctionnalités Clés

  • ✅ Automatisation complète du scraping d'essais - gagnez du temps
  • ✅ Intégration avec Milvus pour un stockage optimisé des données
  • ✅ Enrichissement des données avec OpenAI - améliorez la précision
  • ✅ Utilisation d'agents IA pour interagir efficacement avec les données

📊 Architecture Technique

21
Nodes
14
Connexions
3
Services

🔌 Services Intégrés

n8n-nodes-base.httpRequest@n8n/n8n-nodes-langchain.vectorStoreMilvus@n8n/n8n-nodes-langchain.embeddingsOpenAi

🔧 Composition du Workflow

NodeTypeDescription
When clicking "Execute Workflow"manualTriggerTraitement des données
Fetch Essay ListhttpRequestRequête HTTP vers une API externe
Extract essay nameshtmlTraitement des données
Split out into itemssplitOutDivision des données en plusieurs branches
Fetch essay textshttpRequestRequête HTTP vers une API externe
Limit to first 3limitTraitement des données
Extract Text OnlyhtmlTraitement des données
Sticky Note3stickyNoteTraitement des données
Sticky Note5stickyNoteTraitement des données
Recursive Character Text Splitter1@n8n/n8n-nodes-langchain.textSplitterRecursiveCharacterTextSplitterDivision des données en plusieurs branches
Milvus Vector Store@n8n/n8n-nodes-langchain.vectorStoreMilvusTraitement des données
AI Agent@n8n/n8n-nodes-langchain.agentTraitement des données
When chat message received@n8n/n8n-nodes-langchain.chatTriggerTraitement des données
Sticky NotestickyNoteTraitement des données
Default Data Loader@n8n/n8n-nodes-langchain.documentDefaultDataLoaderTraitement des données
Milvus Vector Store as tool@n8n/n8n-nodes-langchain.vectorStoreMilvusTraitement des données
Sticky Note1stickyNoteTraitement des données
Embeddings OpenAI@n8n/n8n-nodes-langchain.embeddingsOpenAiTraitement des données
OpenAI Chat Model@n8n/n8n-nodes-langchain.lmChatOpenAiTraitement des données
Embeddings OpenAI1@n8n/n8n-nodes-langchain.embeddingsOpenAiTraitement des données
Sticky Note2stickyNoteTraitement des données

📖 Guide d'Implémentation

  1. Import du workflow: Téléchargez le fichier JSON et importez-le dans votre instance n8n
  2. Configuration des credentials: Configurez les accès pour chaque service utilisé
  3. Personnalisation: Adaptez les paramètres selon vos besoins spécifiques
  4. Test: Exécutez le workflow en mode test pour vérifier le bon fonctionnement
  5. Activation: Activez le workflow pour une exécution automatique

🏷️ Tags

ScrapingMilvusOpenAI

Structure JSON

Voir le code JSON complet
{
    "id": "A5R7XYSzrCJKlw9k",
    "meta": {
        "instanceId": "2c4c1e23e7b067270c08aab616bada21d0c384d16f212b23cf1143c6baa09219",
        "templateCredsSetupCompleted": true
    },
    "name": "Agent Milvus tool",
    "tags": [
        {
            "id": "msnDWKHQmwMDxWQH",
            "name": "Milvus",
            "createdAt": "2025-04-16T12:48:14.539Z",
            "updatedAt": "2025-04-16T12:48:14.539Z"
        },
        {
            "id": "tnCpo8hq8uKrdASK",
            "name": "AI",
            "createdAt": "2025-04-16T12:47:57.976Z",
            "updatedAt": "2025-04-16T12:47:57.976Z"
        }
    ],
    "nodes": [
        {
            "id": "cfe6264a-2be1-4d1e-974b-ee05ca8ae9ab",
            "name": "When clicking \"Execute Workflow\"",
            "type": "n8n-nodes-base.manualTrigger",
            "position": [
                -280,
                -40
            ],
            "parameters": [],
            "typeVersion": 1
        },
        {
            "id": "c0665cc9-2bce-48db-a3bc-15baac68e569",
            "name": "Fetch Essay List",
            "type": "n8n-nodes-base.httpRequest",
            "position": [
                -20,
                -40
            ],
            "parameters": {
                "url": "http:\/\/www.paulgraham.com\/articles.html",
                "options": []
            },
            "typeVersion": 4.2
        },
        {
            "id": "00bcdc0b-eb6d-41eb-ac0d-a6710d6232e4",
            "name": "Extract essay names",
            "type": "n8n-nodes-base.html",
            "position": [
                180,
                -40
            ],
            "parameters": {
                "options": [],
                "operation": "extractHtmlContent",
                "extractionValues": {
                    "values": [
                        {
                            "key": "essay",
                            "attribute": "href",
                            "cssSelector": "table table a",
                            "returnArray": true,
                            "returnValue": "attribute"
                        }
                    ]
                }
            },
            "typeVersion": 1.2
        },
        {
            "id": "523c319e-d1c7-4214-a725-dc557f6471a2",
            "name": "Split out into items",
            "type": "n8n-nodes-base.splitOut",
            "position": [
                380,
                -40
            ],
            "parameters": {
                "options": [],
                "fieldToSplitOut": "essay"
            },
            "typeVersion": 1
        },
        {
            "id": "be155368-99f5-43b3-ba6c-50cccf2b72d2",
            "name": "Fetch essay texts",
            "type": "n8n-nodes-base.httpRequest",
            "position": [
                780,
                -40
            ],
            "parameters": {
                "url": "=http:\/\/www.paulgraham.com\/{{ $json.essay }}",
                "options": []
            },
            "typeVersion": 4.2
        },
        {
            "id": "92af113c-dd71-4ddd-b50a-f5932392ed82",
            "name": "Limit to first 3",
            "type": "n8n-nodes-base.limit",
            "position": [
                580,
                -40
            ],
            "parameters": {
                "maxItems": 3
            },
            "typeVersion": 1
        },
        {
            "id": "1a1893c4-e8b2-454a-b49f-a0b0f3c01aca",
            "name": "Extract Text Only",
            "type": "n8n-nodes-base.html",
            "position": [
                1100,
                -40
            ],
            "parameters": {
                "options": [],
                "operation": "extractHtmlContent",
                "extractionValues": {
                    "values": [
                        {
                            "key": "data",
                            "cssSelector": "body",
                            "skipSelectors": "img,nav"
                        }
                    ]
                }
            },
            "typeVersion": 1.2
        },
        {
            "id": "d14ae606-f002-4fde-a896-bf1c7fa675b2",
            "name": "Sticky Note3",
            "type": "n8n-nodes-base.stickyNote",
            "position": [
                -100,
                -160
            ],
            "parameters": {
                "width": 1071.752021563343,
                "height": 285.66037735849045,
                "content": "## Scrape latest Paul Graham essays"
            },
            "typeVersion": 1
        },
        {
            "id": "dfb0cb32-9d7c-4588-b75e-0b79231eb72a",
            "name": "Sticky Note5",
            "type": "n8n-nodes-base.stickyNote",
            "position": [
                1020,
                -160
            ],
            "parameters": {
                "width": 625,
                "height": 607,
                "content": "## Load into Milvus vector database"
            },
            "typeVersion": 1
        },
        {
            "id": "862a1a02-50e2-42af-9fa9-eb3a4f2ca463",
            "name": "Recursive Character Text Splitter1",
            "type": "@n8n\/n8n-nodes-langchain.textSplitterRecursiveCharacterTextSplitter",
            "position": [
                1440,
                300
            ],
            "parameters": {
                "options": [],
                "chunkSize": 6000
            },
            "typeVersion": 1
        },
        {
            "id": "91ac110a-57db-44b1-b22f-d2a63f22f173",
            "name": "Milvus Vector Store",
            "type": "@n8n\/n8n-nodes-langchain.vectorStoreMilvus",
            "position": [
                1320,
                -40
            ],
            "parameters": {
                "mode": "insert",
                "options": {
                    "clearCollection": true
                },
                "milvusCollection": {
                    "__rl": true,
                    "mode": "list",
                    "value": "n8n_test",
                    "cachedResultName": "n8n_test"
                }
            },
            "credentials": {
                "milvusApi": {
                    "id": "8tMHHoLiWXIAXa7S",
                    "name": "Milvus account"
                }
            },
            "typeVersion": 1.1
        },
        {
            "id": "456e917f-d466-4ec8-8df9-3774ba58151d",
            "name": "AI Agent",
            "type": "@n8n\/n8n-nodes-langchain.agent",
            "position": [
                60,
                360
            ],
            "parameters": {
                "options": []
            },
            "typeVersion": 1.9
        },
        {
            "id": "a5c5f308-097d-4fe0-92be-d717fd1e0b74",
            "name": "When chat message received",
            "type": "@n8n\/n8n-nodes-langchain.chatTrigger",
            "position": [
                -280,
                360
            ],
            "webhookId": "cd2703a7-f912-46fe-8787-3fb83ea116ab",
            "parameters": {
                "options": []
            },
            "typeVersion": 1.1
        },
        {
            "id": "dc352f07-335f-47cb-8270-32a4a0b87df7",
            "name": "Sticky Note",
            "type": "n8n-nodes-base.stickyNote",
            "position": [
                -460,
                -200
            ],
            "parameters": {
                "width": 280,
                "height": 180,
                "content": "## Step 1\n1. Set up a Milvus server based on [this guide](https:\/\/milvus.io\/docs\/install_standalone-docker-compose.md). And then create a collection named `n8n_test`.\n2. Click this workflow to load scrape and load Paul Graham essays to Milvus collection.\n"
            },
            "typeVersion": 1
        },
        {
            "id": "5c9e9871-c9c1-458e-b35c-eab87ac5ca26",
            "name": "Default Data Loader",
            "type": "@n8n\/n8n-nodes-langchain.documentDefaultDataLoader",
            "position": [
                1360,
                180
            ],
            "parameters": {
                "options": [],
                "jsonData": "={{ $('Extract Text Only').item.json.data }}",
                "jsonMode": "expressionData"
            },
            "typeVersion": 1
        },
        {
            "id": "5b202001-525c-4481-a263-56b69c9b1bd8",
            "name": "Milvus Vector Store as tool",
            "type": "@n8n\/n8n-nodes-langchain.vectorStoreMilvus",
            "position": [
                180,
                560
            ],
            "parameters": {
                "mode": "retrieve-as-tool",
                "toolName": "milvus_knowledge_base",
                "toolDescription": "useful when you need to retrieve information",
                "milvusCollection": {
                    "__rl": true,
                    "mode": "list",
                    "value": "n8n_test",
                    "cachedResultName": "n8n_test"
                }
            },
            "credentials": {
                "milvusApi": {
                    "id": "8tMHHoLiWXIAXa7S",
                    "name": "Milvus account"
                }
            },
            "typeVersion": 1.1
        },
        {
            "id": "6b5b95c7-dde2-4c3f-952b-97a8f5c267c9",
            "name": "Sticky Note1",
            "type": "n8n-nodes-base.stickyNote",
            "position": [
                -460,
                260
            ],
            "parameters": {
                "width": 280,
                "height": 120,
                "content": "## Step 2\nStart to chat with the AI Agent with Milvus tool"
            },
            "typeVersion": 1
        },
        {
            "id": "5ccfe636-2bb3-4026-98f0-57ba8d5780f0",
            "name": "Embeddings OpenAI",
            "type": "@n8n\/n8n-nodes-langchain.embeddingsOpenAi",
            "position": [
                1220,
                200
            ],
            "parameters": {
                "options": []
            },
            "credentials": {
                "openAiApi": {
                    "id": "hH2PTDH4fbS7fdPv",
                    "name": "OpenAi account"
                }
            },
            "typeVersion": 1.2
        },
        {
            "id": "982622e9-af05-4ee2-ae7d-166c47f75ce9",
            "name": "OpenAI Chat Model",
            "type": "@n8n\/n8n-nodes-langchain.lmChatOpenAi",
            "position": [
                20,
                560
            ],
            "parameters": {
                "model": {
                    "__rl": true,
                    "mode": "list",
                    "value": "gpt-4o-mini"
                },
                "options": []
            },
            "credentials": {
                "openAiApi": {
                    "id": "hH2PTDH4fbS7fdPv",
                    "name": "OpenAi account"
                }
            },
            "typeVersion": 1.2
        },
        {
            "id": "abd97878-cce6-44a0-8bae-91536ea48b6b",
            "name": "Embeddings OpenAI1",
            "type": "@n8n\/n8n-nodes-langchain.embeddingsOpenAi",
            "position": [
                200,
                740
            ],
            "parameters": {
                "options": []
            },
            "credentials": {
                "openAiApi": {
                    "id": "hH2PTDH4fbS7fdPv",
                    "name": "OpenAi account"
                }
            },
            "typeVersion": 1.2
        },
        {
            "id": "00d49aab-3200-44fc-a0fc-8f7f22998617",
            "name": "Sticky Note2",
            "type": "n8n-nodes-base.stickyNote",
            "position": [
                -80,
                300
            ],
            "parameters": {
                "color": 7,
                "width": 574,
                "height": 629,
                "content": ""
            },
            "typeVersion": 1
        }
    ],
    "active": false,
    "pinData": [],
    "settings": {
        "executionOrder": "v1"
    },
    "versionId": "8e6f0bb5-1fb5-48fc-8a1f-488362be4ef7",
    "connections": {
        "Fetch Essay List": {
            "main": [
                [
                    {
                        "node": "Extract essay names",
                        "type": "main",
                        "index": 0
                    }
                ]
            ]
        },
        "Limit to first 3": {
            "main": [
                [
                    {
                        "node": "Fetch essay texts",
                        "type": "main",
                        "index": 0
                    }
                ]
            ]
        },
        "Embeddings OpenAI": {
            "ai_embedding": [
                [
                    {
                        "node": "Milvus Vector Store",
                        "type": "ai_embedding",
                        "index": 0
                    }
                ]
            ]
        },
        "Extract Text Only": {
            "main": [
                [
                    {
                        "node": "Milvus Vector Store",
                        "type": "main",
                        "index": 0
                    }
                ]
            ]
        },
        "Fetch essay texts": {
            "main": [
                [
                    {
                        "node": "Extract Text Only",
                        "type": "main",
                        "index": 0
                    }
                ]
            ]
        },
        "OpenAI Chat Model": {
            "ai_languageModel": [
                [
                    {
                        "node": "AI Agent",
                        "type": "ai_languageModel",
                        "index": 0
                    }
                ]
            ]
        },
        "Embeddings OpenAI1": {
            "ai_embedding": [
                [
                    {
                        "node": "Milvus Vector Store as tool",
                        "type": "ai_embedding",
                        "index": 0
                    }
                ]
            ]
        },
        "Default Data Loader": {
            "ai_document": [
                [
                    {
                        "node": "Milvus Vector Store",
                        "type": "ai_document",
                        "index": 0
                    }
                ]
            ]
        },
        "Extract essay names": {
            "main": [
                [
                    {
                        "node": "Split out into items",
                        "type": "main",
                        "index": 0
                    }
                ]
            ]
        },
        "Split out into items": {
            "main": [
                [
                    {
                        "node": "Limit to first 3",
                        "type": "main",
                        "index": 0
                    }
                ]
            ]
        },
        "When chat message received": {
            "main": [
                [
                    {
                        "node": "AI Agent",
                        "type": "main",
                        "index": 0
                    }
                ]
            ]
        },
        "Milvus Vector Store as tool": {
            "ai_tool": [
                [
                    {
                        "node": "AI Agent",
                        "type": "ai_tool",
                        "index": 0
                    }
                ]
            ]
        },
        "When clicking \"Execute Workflow\"": {
            "main": [
                [
                    {
                        "node": "Fetch Essay List",
                        "type": "main",
                        "index": 0
                    }
                ]
            ]
        },
        "Recursive Character Text Splitter1": {
            "ai_textSplitter": [
                [
                    {
                        "node": "Default Data Loader",
                        "type": "ai_textSplitter",
                        "index": 0
                    }
                ]
            ]
        }
    }
}
                                

Workflows Similaires

Comprehensive n8n Creator Stats Automation Workflow

Automate the reporting of top n8n creators and workflows with this powerful workflow. By aggregating data from GitHub, g...

Analyse Automatisée des États Américains par l'IA

Ce workflow n8n permet d'analyser automatiquement les plus grands états des USA en termes de superficie, en listant leu...

Automatisez l'import de CSV vers Excel en toute simplicité

Ce workflow n8n simplifie la conversion de fichiers CSV en fichiers Excel (.xlsx), un processus essentiel pour les profe...