使用Qdrant和通用在线LMs, 为您的博客“-” 建造“"LAG” (中文 (Chinese Simplified))

一. 导言导言导言导言导言

在我的 GPT="系列律师,我教你如何使用 GPU 加速度@,当地 LLMZ,和矢量数据库 *I.ItMS K5s power_,private{,}建立完整的本地 RAG}-基写作助理,并完全在您的硬件上运行.

但不是每个人都有一个工作站, RAM=,}的NVIDIA GPU,+ 96QGB 以及建立CUDA+,和拼接GGGUF Model_.的耐心。如果你想要一个没有硬件投资的博客写作助理的好处呢?

本篇文章展示了基于云端的 -基选项@:同样的 RAG 方法 @,相同的 Qdrant矢量数据库,,但使用云端 LLM APIs而不是本地推论 *. 把它想象成“"律师 GPT Lite"}

完全披露@: I'm 仍在学习哪种方法在实践上最有效 @, 所以用我的成本估算和业绩申报用一点盐来表示吧我只能说,与GPU路线相比这种云计算法非常简单

这是我自己编辑的A )Q MS K3Q 同一声音同样的实用性 ,Q 一样

为何云源可替代 @? @

原始方法

完整的“"”律师GPT+"系列建立了一个系统,这个系统是“MSC2”建立的。

本地运行 @100% @ @ ( @ privacy@ MS K2}
无APPI费用
快速 GPU- 加速发酵
需要 NVIDIA GPU @ (8 @ GB=+} VRAM)
复杂设置@(CUDA, cudn,模式管理_)
限为您可在 VRAM 中适合的模型
Windows - 重心部署

云的替代品

此方法给予您@ : @%

在任何机器上没有 GPU 要求“ ( @ runs” )
简易设置 @ (no CUDA/cuDNN})
访问边境模型@(GPT-4, Claude},等
横跨“-”平台格式 “(”窗口 @,}Macó, Linux@)
更好的输出质量 @ ( @ grandr@ MS K1} 更有能力的模型 @ MPK2 @
API成本( 虽然个人使用合理
数据已发送到第三方@-_Party APIs
延迟取决于网络

何时使用什么@ ? @

@| 使用本地方法@| 利用云端方法 *| |--------------------|-------------------| 隐私是关键 | 您有 GPU 硬件 @|}您在 MacQ/Linux\ /pattop *| @| 高容量使用@| 中度使用 @MS K2 few posts @/ month_ ) @#| |你喜欢修补木匠 MS K1 你想要结果快

建筑结构概览

云端版本保留相同的 RAG 基本原理, 但将本地 LLM 推论换成 API 调用@:

graph TB
    A[Markdown Files] -->|Ingest| B[Chunking Service]
    B -->|Text Chunks| C[Cloud Embedding API]
    C -->|Vectors| D[Qdrant Vector DB]

    E[User Writing] -->|Current Draft| F[Web/Desktop Client]
    F -->|Embed Context| C
    C -->|Query Vector| D
    D -->|Similar Content| G[Context Builder]

    G -->|Relevant Past Articles| H[Prompt Engineer]
    H -->|Prompt + Context| I[Cloud LLM API]
    I -->|Generated Suggestions| J[Response Handler]
    J -->|Suggestions + Citations| F

    F -->|Display| K[Editor with Suggestions]

    class C,I cloud
    class D,K local

    classDef cloud stroke:#f96,stroke-width:4px
    classDef local stroke:#333,stroke-width:2px

密钥差异@: @%

嵌入模型@ : @ OpenAI' @%s text-embedding-3-small API而不是当地BGE模式
法学硕士“: Claude”“3.5”Sonnet 或 GPT-4 API 而不是本地 Mistral}/Llama
不需要 GPU:所有基于本地的 CPU-@, 计算在云中发生
简化部署: 单一可执行程序 @, 没有要管理的示范文件@ info

我要指出的是,我曾经有 't 运行广泛的基准比较两种方法而目前还在探索阶段的I- IMSC2m

技术堆叠

核心框架

# .NET @9# - 与原序列相同
C# □13 现代语言特写

云层APPs

O开放AI API - 嵌入式(Text-}MS K3Small)}MPK5LLM{(GPT}-4)
人类APIPI - 替代LLM @(#[email protected]_Sonnet})
两者你可以混合和匹配 !

矢量数据库

解冻 - 和原始@,一样, 可以本地通过 Docker 运行或使用 Qdrant Cloud
替代@:菠萝 , 湿云 I( 管理选项)

客户选项

控制台程序 -=YTET -伊甸园字幕组=- 翻译:
布拉佐尔网络大会 @-Web@-Based_,在任何地方工作
瓦隆 - 十字 MS K1 平台桌面 @( Windows, Mac, Linux})

设置“快速路径”

1.安裝 Qdrant

选项 A: 本地 docker ( 建议发展_)

docker run -p 6333:6333 -p 6334:6334 \
    -v $(pwd)/qdrant_storage:/qdrant/storage:z \
    qdrant/qdrant

选项 B: 倾斜云 ( EasiestMS K2

注册时间云端 .qdrant.io
创建一个免费集创建
获取您的 API 密钥和聚集 URL

没有 CUDA,没有 cudNNMS K1 不需要任何驱动器装置\ !

2. 获得API 密钥

开放国际 用于嵌入的({+}LLM=):

转到平台@._openai @._com
创建 API 密钥
设定使用限制@ ( @ importal @ MS K1

人类 Claude=):的,

转到控制台: . @ humanic_ . @ com
创建 API 密钥

@ 3._ 配置

创建创建 appsettings.json:

{
  "BlogRAG": {
    "Embedding": {
      "Provider": "OpenAI",
      "Model": "text-embedding-3-small",
      "ApiKey": "sk-..."
    },
    "LLM": {
      "Provider": "Anthropic",
      "Model": "claude-3-5-sonnet-20241022",
      "ApiKey": "sk-ant-..."
    },
    "VectorStore": {
      "Type": "Qdrant",
      "Url": "http://localhost:6333",
      "ApiKey": "",
      "CollectionName": "blog_embeddings"
    },
    "Ingestion": {
      "MarkdownPath": "/path/to/your/blog/Markdown",
      "ChunkSize": 500,
      "ChunkOverlap": 50
    }
  }
}

万岁,万岁 没有 GPU 设置@,}没有模式下载 *\ (12 GB 文件 @),}无 VRAM 管理=.}

执行执行情况

核心服务

1.云嵌入服务

using OpenAI;
using OpenAI.Embeddings;

namespace BlogRAG.Services
{
    public interface IEmbeddingService
    {
        Task<float[]> GenerateEmbeddingAsync(string text);
        Task<List<float[]>> GenerateBatchEmbeddingsAsync(List<string> texts);
    }

    public class OpenAIEmbeddingService : IEmbeddingService
    {
        private readonly OpenAIClient _client;
        private readonly string _model;
        private readonly ILogger<OpenAIEmbeddingService> _logger;

        public OpenAIEmbeddingService(
            string apiKey,
            string model,
            ILogger<OpenAIEmbeddingService> logger)
        {
            _client = new OpenAIClient(apiKey);
            _model = model;
            _logger = logger;
        }

        public async Task<float[]> GenerateEmbeddingAsync(string text)
        {
            var embeddings = await GenerateBatchEmbeddingsAsync(new List<string> { text });
            return embeddings.First();
        }

        public async Task<List<float[]>> GenerateBatchEmbeddingsAsync(List<string> texts)
        {
            _logger.LogInformation("Generating embeddings for {Count} texts", texts.Count);

            var request = new EmbeddingRequest
            {
                Input = texts,
                Model = _model
            };

            var response = await _client.CreateEmbeddingAsync(request);

            return response.Data
                .OrderBy(e => e.Index)
                .Select(e => e.Embedding.ToArray())
                .ToList();
        }
    }
}

关键利益相对于本地端@: @%

无 ONNX 运行时间设置
无 GPU 内存管理
OpenAI 自动分批
状态@ -_ of @ - @ the MS K2# 艺术嵌入质量

费用成本1K的标语非常便宜

正在处理 @100博客文章@(_{500Ksigns @):}#}$0.05
每日使用@(10_问询 @): _~$0.001/日 @MS K3}{$0.30/个月

* 云层LLM服务

using Anthropic.SDK;
using Anthropic.SDK.Messaging;

namespace BlogRAG.Services
{
    public interface ILLMService
    {
        Task<string> GenerateCompletionAsync(
            string systemPrompt,
            string userPrompt,
            float temperature = 0.7f);

        IAsyncEnumerable<string> GenerateStreamingCompletionAsync(
            string systemPrompt,
            string userPrompt,
            float temperature = 0.7f);
    }

    public class ClaudeLLMService : ILLMService
    {
        private readonly AnthropicClient _client;
        private readonly string _model;
        private readonly ILogger<ClaudeLLMService> _logger;

        public ClaudeLLMService(
            string apiKey,
            string model,
            ILogger<ClaudeLLMService> logger)
        {
            _client = new AnthropicClient(new APIAuthentication(apiKey));
            _model = model;
            _logger = logger;
        }

        public async Task<string> GenerateCompletionAsync(
            string systemPrompt,
            string userPrompt,
            float temperature = 0.7f)
        {
            _logger.LogInformation("Generating completion with temperature {Temp}", temperature);

            var messages = new List<Message>
            {
                new Message
                {
                    Role = RoleType.User,
                    Content = userPrompt
                }
            };

            var request = new MessageRequest
            {
                Model = _model,
                MaxTokens = 2048,
                Temperature = temperature,
                System = systemPrompt,
                Messages = messages
            };

            var response = await _client.Messages.CreateAsync(request);

            return response.Content.First().Text;
        }

        public async IAsyncEnumerable<string> GenerateStreamingCompletionAsync(
            string systemPrompt,
            string userPrompt,
            float temperature = 0.7f)
        {
            var messages = new List<Message>
            {
                new Message { Role = RoleType.User, Content = userPrompt }
            };

            var request = new MessageRequest
            {
                Model = _model,
                MaxTokens = 2048,
                Temperature = temperature,
                System = systemPrompt,
                Messages = messages,
                Stream = true
            };

            await foreach (var chunk in _client.Messages.StreamAsync(request))
            {
                if (chunk.Delta?.Text != null)
                {
                    yield return chunk.Delta.Text;
                }
            }
        }
    }
}

超过本地端的福利金@: @%

没有装入“ ( @ instant 启动” 的模型@ MS K1
没有 VRAM 限制@( @ use @ 200 @ K 上下文( 如果需要的话) @)
更好的输出质量 @ ( @ 至少理论上是 @ I- @ I'# m 还在测试@ ) @
串流运作完美

费用成本:

Claude @3.5_Sonnet@:}$3/#百万个输入代号 @MS K3}%$15/百万个输出
典型的博客写作会话 @(20QK involution\ ,}2QK输出@):}大概是{$0.09}
月使用量 @(10 会话@):}大致是 @MS K2月

这些是基于我早期实验的球场数字 -你的里程数可能因你和AIZ.的闲聊程度而不同

3. Qdrant 矢量存储@(Same 原型!)

using Qdrant.Client;
using Qdrant.Client.Grpc;

namespace BlogRAG.Services
{
    public class QdrantVectorStore
    {
        private readonly QdrantClient _client;
        private readonly string _collectionName;
        private readonly ILogger<QdrantVectorStore> _logger;

        public QdrantVectorStore(
            string url,
            string apiKey,
            string collectionName,
            ILogger<QdrantVectorStore> logger)
        {
            _client = new QdrantClient(url, apiKey: apiKey);
            _collectionName = collectionName;
            _logger = logger;
        }

        public async Task CreateCollectionAsync(int vectorSize)
        {
            var collections = await _client.ListCollectionsAsync();

            if (collections.Any(c => c.Name == _collectionName))
            {
                _logger.LogInformation("Collection {Name} already exists", _collectionName);
                return;
            }

            await _client.CreateCollectionAsync(
                collectionName: _collectionName,
                vectorsConfig: new VectorParams
                {
                    Size = (ulong)vectorSize,
                    Distance = Distance.Cosine
                });

            _logger.LogInformation("Created collection {Name}", _collectionName);
        }

        public async Task UpsertAsync(
            Guid id,
            float[] vector,
            Dictionary<string, object> payload)
        {
            var point = new PointStruct
            {
                Id = id,
                Vectors = vector,
                Payload = payload
            };

            await _client.UpsertAsync(_collectionName, new[] { point });
        }

        public async Task<List<ScoredPoint>> SearchAsync(
            float[] queryVector,
            int limit = 10,
            float scoreThreshold = 0.7f)
        {
            var results = await _client.SearchAsync(
                collectionName: _collectionName,
                vector: queryVector,
                limit: (ulong)limit,
                scoreThreshold: scoreThreshold);

            return results.ToList();
        }
    }
}

与本地设置相同的 API - 指向本地的 Docker 或 Qdrant Cloud!

摄取管道

namespace BlogRAG.Services
{
    public class IngestionService
    {
        private readonly IEmbeddingService _embedder;
        private readonly QdrantVectorStore _vectorStore;
        private readonly ILogger<IngestionService> _logger;

        public IngestionService(
            IEmbeddingService embedder,
            QdrantVectorStore vectorStore,
            ILogger<IngestionService> logger)
        {
            _embedder = embedder;
            _vectorStore = vectorStore;
            _logger = logger;
        }

        public async Task IngestMarkdownFilesAsync(string markdownPath)
        {
            var files = Directory.GetFiles(markdownPath, "*.md", SearchOption.AllDirectories);
            _logger.LogInformation("Found {Count} markdown files", files.Length);

            foreach (var file in files)
            {
                await IngestFileAsync(file);
            }
        }

        private async Task IngestFileAsync(string filePath)
        {
            var content = await File.ReadAllTextAsync(filePath);
            var metadata = ExtractMetadata(content);
            var chunks = ChunkContent(content);

            _logger.LogInformation("Processing {File}: {ChunkCount} chunks",
                Path.GetFileName(filePath), chunks.Count);

            // Batch embedding generation
            var texts = chunks.Select(c => c.Text).ToList();
            var embeddings = await _embedder.GenerateBatchEmbeddingsAsync(texts);

            // Upload to Qdrant
            for (int i = 0; i < chunks.Count; i++)
            {
                var chunk = chunks[i];
                var embedding = embeddings[i];

                var payload = new Dictionary<string, object>
                {
                    ["text"] = chunk.Text,
                    ["file_path"] = filePath,
                    ["blog_post_slug"] = metadata.Slug,
                    ["blog_post_title"] = metadata.Title,
                    ["chunk_index"] = i,
                    ["category"] = metadata.Category
                };

                await _vectorStore.UpsertAsync(Guid.NewGuid(), embedding, payload);
            }

            _logger.LogInformation("Ingested {File}", Path.GetFileName(filePath));
        }

        private List<TextChunk> ChunkContent(string content, int chunkSize = 500, int overlap = 50)
        {
            // Simple sentence-aware chunking
            var sentences = content.Split(new[] { ". ", ".\n", "!\n", "?\n" },
                StringSplitOptions.RemoveEmptyEntries);

            var chunks = new List<TextChunk>();
            var currentChunk = new StringBuilder();
            var currentLength = 0;

            foreach (var sentence in sentences)
            {
                if (currentLength + sentence.Length > chunkSize && currentChunk.Length > 0)
                {
                    chunks.Add(new TextChunk { Text = currentChunk.ToString() });

                    // Overlap: keep last sentence
                    currentChunk.Clear();
                    currentLength = 0;
                }

                currentChunk.Append(sentence).Append(". ");
                currentLength += sentence.Length;
            }

            if (currentChunk.Length > 0)
            {
                chunks.Add(new TextChunk { Text = currentChunk.ToString() });
            }

            return chunks;
        }

        private BlogMetadata ExtractMetadata(string content)
        {
            // Extract from markdown frontmatter or HTML comments
            var titleMatch = Regex.Match(content, @"^#\s+(.+)$", RegexOptions.Multiline);
            var categoryMatch = Regex.Match(content, @"");

            return new BlogMetadata
            {
                Title = titleMatch.Success ? titleMatch.Groups[1].Value : "Untitled",
                Category = categoryMatch.Success ? categoryMatch.Groups[1].Value : "General",
                Slug = Path.GetFileNameWithoutExtension(content)
            };
        }
    }

    public class TextChunk
    {
        public string Text { get; set; } = string.Empty;
    }

    public class BlogMetadata
    {
        public string Title { get; set; } = string.Empty;
        public string Category { get; set; } = string.Empty;
        public string Slug { get; set; } = string.Empty;
    }
}

RAG 生成处

namespace BlogRAG.Services
{
    public class RAGGenerationService
    {
        private readonly IEmbeddingService _embedder;
        private readonly QdrantVectorStore _vectorStore;
        private readonly ILLMService _llm;
        private readonly ILogger<RAGGenerationService> _logger;

        public RAGGenerationService(
            IEmbeddingService embedder,
            QdrantVectorStore vectorStore,
            ILLMService llm,
            ILogger<RAGGenerationService> logger)
        {
            _embedder = embedder;
            _vectorStore = vectorStore;
            _llm = llm;
            _logger = logger;
        }

        public async Task<string> GenerateSuggestionAsync(
            string currentDraft,
            string requestType = "continue")
        {
            // 1. Generate embedding for current draft
            var draftEmbedding = await _embedder.GenerateEmbeddingAsync(currentDraft);

            // 2. Search for relevant past content
            var results = await _vectorStore.SearchAsync(
                queryVector: draftEmbedding,
                limit: 5,
                scoreThreshold: 0.7f);

            _logger.LogInformation("Found {Count} relevant chunks", results.Count);

            // 3. Build context from results
            var contextBuilder = new StringBuilder();
            foreach (var result in results)
            {
                var text = result.Payload["text"].ToString();
                var title = result.Payload["blog_post_title"].ToString();
                var score = result.Score;

                contextBuilder.AppendLine($"## From: {title} (relevance: {score:F2})");
                contextBuilder.AppendLine(text);
                contextBuilder.AppendLine();
            }

            // 4. Build prompt
            var systemPrompt = BuildSystemPrompt(requestType);
            var userPrompt = BuildUserPrompt(currentDraft, contextBuilder.ToString(), requestType);

            // 5. Generate with LLM
            var suggestion = await _llm.GenerateCompletionAsync(
                systemPrompt: systemPrompt,
                userPrompt: userPrompt,
                temperature: 0.7f);

            return suggestion;
        }

        private string BuildSystemPrompt(string requestType)
        {
            return requestType switch
            {
                "continue" => @"You are a technical blog writing assistant. Your role is to suggest
                    continuations for blog posts based on the author's past writing style and content.

                    Guidelines:
                    - Match the author's voice and technical depth
                    - Use similar patterns and structures from past posts
                    - Be specific and technical, not generic
                    - Include code examples when relevant
                    - Maintain consistency with past content",

                "improve" => @"You are a technical blog editor. Your role is to improve sections
                    of blog posts while maintaining the author's voice.

                    Guidelines:
                    - Preserve the author's style
                    - Improve clarity and flow
                    - Add technical depth where appropriate
                    - Suggest better examples from past posts
                    - Fix unclear explanations",

                "outline" => @"You are a technical blog outline generator. Your role is to suggest
                    outlines for new blog posts based on past structures.

                    Guidelines:
                    - Study the author's typical post structure
                    - Suggest sections based on successful past posts
                    - Include technical depth appropriate to topic
                    - Reference similar past articles",

                _ => "You are a helpful technical writing assistant."
            };
        }

        private string BuildUserPrompt(string currentDraft, string context, string requestType)
        {
            return $@"
# Current Draft
{currentDraft}

# Relevant Past Content
{context}

# Request
{GetRequestDescription(requestType)}

Please provide your suggestion based on the current draft and the relevant past content shown above.
Remember to maintain consistency with the author's past writing style and technical approach.
";
        }

        private string GetRequestDescription(string requestType)
        {
            return requestType switch
            {
                "continue" => "Continue writing from where the draft ends. Suggest the next 1-2 paragraphs.",
                "improve" => "Improve the current draft. Suggest specific edits and enhancements.",
                "outline" => "Create a detailed outline for completing this post.",
                _ => "Provide helpful suggestions."
            };
        }
    }
}

简单控制台客户端

using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;

namespace BlogRAG.Console
{
    class Program
    {
        static async Task Main(string[] args)
        {
            // Setup DI and configuration
            var services = new ServiceCollection();

            var configuration = new ConfigurationBuilder()
                .SetBasePath(Directory.GetCurrentDirectory())
                .AddJsonFile("appsettings.json")
                .AddUserSecrets<Program>()  // For API keys
                .Build();

            services.AddLogging(builder => builder.AddConsole());

            // Register services
            var embeddingConfig = configuration.GetSection("BlogRAG:Embedding");
            services.AddSingleton<IEmbeddingService>(sp =>
                new OpenAIEmbeddingService(
                    embeddingConfig["ApiKey"]!,
                    embeddingConfig["Model"]!,
                    sp.GetRequiredService<ILogger<OpenAIEmbeddingService>>()));

            var llmConfig = configuration.GetSection("BlogRAG:LLM");
            services.AddSingleton<ILLMService>(sp =>
                new ClaudeLLMService(
                    llmConfig["ApiKey"]!,
                    llmConfig["Model"]!,
                    sp.GetRequiredService<ILogger<ClaudeLLMService>>()));

            var vectorConfig = configuration.GetSection("BlogRAG:VectorStore");
            services.AddSingleton(sp =>
                new QdrantVectorStore(
                    vectorConfig["Url"]!,
                    vectorConfig["ApiKey"] ?? "",
                    vectorConfig["CollectionName"]!,
                    sp.GetRequiredService<ILogger<QdrantVectorStore>>()));

            services.AddSingleton<IngestionService>();
            services.AddSingleton<RAGGenerationService>();

            var serviceProvider = services.BuildServiceProvider();

            // Run CLI
            await RunCLI(serviceProvider, configuration);
        }

        static async Task RunCLI(ServiceProvider serviceProvider, IConfiguration configuration)
        {
            System.Console.WriteLine("=== Blog RAG Assistant ===\n");
            System.Console.WriteLine("Commands:");
            System.Console.WriteLine("  ingest - Ingest markdown files");
            System.Console.WriteLine("  write - Start writing session");
            System.Console.WriteLine("  quit - Exit\n");

            while (true)
            {
                System.Console.Write("> ");
                var command = System.Console.ReadLine()?.Trim().ToLower();

                switch (command)
                {
                    case "ingest":
                        await IngestCommand(serviceProvider, configuration);
                        break;
                    case "write":
                        await WriteCommand(serviceProvider);
                        break;
                    case "quit":
                        return;
                    default:
                        System.Console.WriteLine("Unknown command");
                        break;
                }
            }
        }

        static async Task IngestCommand(ServiceProvider serviceProvider, IConfiguration configuration)
        {
            var ingestion = serviceProvider.GetRequiredService<IngestionService>();
            var markdownPath = configuration["BlogRAG:Ingestion:MarkdownPath"];

            System.Console.WriteLine($"Ingesting from {markdownPath}...");
            await ingestion.IngestMarkdownFilesAsync(markdownPath!);
            System.Console.WriteLine("Ingestion complete!\n");
        }

        static async Task WriteCommand(ServiceProvider serviceProvider)
        {
            var rag = serviceProvider.GetRequiredService<RAGGenerationService>();

            System.Console.WriteLine("\nEnter your draft (end with empty line):");
            var draft = new StringBuilder();
            string? line;

            while (!string.IsNullOrWhiteSpace(line = System.Console.ReadLine()))
            {
                draft.AppendLine(line);
            }

            System.Console.WriteLine("\nGenerating suggestion...\n");
            var suggestion = await rag.GenerateSuggestionAsync(draft.ToString());

            System.Console.WriteLine("=== Suggestion ===");
            System.Console.WriteLine(suggestion);
            System.Console.WriteLine("\n");
        }
    }
}

运行系统

首次时间设置

# 1. Clone/create project
dotnet new console -n BlogRAG
cd BlogRAG

# 2. Add packages
dotnet add package Qdrant.Client
dotnet add package OpenAI
dotnet add package Anthropic.SDK
dotnet add package Microsoft.Extensions.Configuration.Json
dotnet add package Microsoft.Extensions.Configuration.UserSecrets

# 3. Set API keys (stored securely)
dotnet user-secrets init
dotnet user-secrets set "BlogRAG:Embedding:ApiKey" "sk-..."
dotnet user-secrets set "BlogRAG:LLM:ApiKey" "sk-ant-..."

# 4. Start Qdrant (local)
docker run -d -p 6333:6333 qdrant/qdrant

# 5. Run ingestion
dotnet run
> ingest

# 6. Start writing
> write

设置总时间@:@ 大致是 @MS K1 @ 分钟与当地GPU设置的当地 GPU时间大约为#2_ 小时@-}假设一切顺利,

每日使用量

# Start Qdrant (if using local Docker)
docker start qdrant

# Run assistant
dotnet run
> write

# Enter your draft
I've been working on a new feature that uses Entity Framework Core...
[Ctrl+D or empty line]

# Get AI suggestion based on your past EF posts!

成本分析

每月成本估算@( @personalblog_)

假设情景每月#:写@4博客文章

-=YTET -伊甸园字幕组=- 翻译: |-----------|--------|------| @| @(100_BARBAR_ )}|_One@-Time{,}#500_K标语 @MS K7_QZK8|} @| 嵌入符號@(Q}, 40/ month\ MS K4 NSK5 I40 标语#| #$0.004~| “|”LLM呼叫“(40”建议 ) @| 800K输入=,#80K输出 @MS K7QZ$3.60~| | 月每月共计 | | ~$3.65 |

用于比较@: @%

本地设置@: @ @ $0/_ 月度 @(# but *$800+} GPU 在前端 @ MPK4
@$20/_月 (_no RAG,}通用@)
@$12/_月 #(No AI write@)#

断点@- @ 偶点:+@'+#如果您用这个来做 @18+_% months @MS K3}本地GPU 为自己支付$.+% 否则 ,+%云是更便宜的 #.+#虽然I'+%仍然在研究我的成本预测是否准确

费用优化提示

使用较小的嵌入模型:
- text-embedding-3-small: @ @ MS K1QK 象征性牌
- text-embedding-3-large: @ @ MS K1QK 象征性牌
- -=YTET -伊甸园字幕组=- 翻译:

批次APP电话 “(50%”对非“-”来说便宜一些。

// Use OpenAI Batch API for ingestion
var batch = await client.CreateBatchAsync(requests);
// Wait hours, pay half price

本地缓存嵌入:

// Don't re-embed identical text
var cache = new Dictionary<string, float[]>();

采用更便宜的草案模式:
- Claude @3.5 @Haiku@:}$0.25/#M 输入 *(12#x 比Sonnet#)#便宜
- GPT (20x 比GPTQ-4)便宜

限制上下文窗口:

// Retrieve top 3 instead of top 10 chunks
limit: 3  // 70% less input tokens

超越本地设置的有利条件

更好的样板质量

模范 @	上下文@	QQQ}本地	? @MS K5云?#
@	}Yes	需要#( 需要$(8+GB VRAMQ))	#是□	#
@8_K5} 超棒的	#没有(需要 *48GB VRAMQ)}{	}是□	*
GPT-4Turbo	MSSK3K □	精美的 @	否	是#
{\fn黑体\fs22\bord1\shad0\3aHBE\4aH00\fscx67\fscy66\2cHFFFFFF\3cH808080}	克劳德 Claude 3.5 索内特 MS K2 200 最佳

Cloud 提供您访问“ 70” B+ 模型的机会 这需要 GPU 硬件%.}至少是,}这理论认为' "I-" IQ'm 仍在学习大模型实际上是否在实际中产生了明显更好的博客内容吗?.

2.即时更新

// Switch models with one line
services.AddSingleton<ILLMService>(sp =>
    new ClaudeLLMService(  // Was GPT-4, now Claude
        config["ApiKey"],
        "claude-3-5-sonnet-20241022",  // Latest model
        sp.GetRequiredService<ILogger<ClaudeLLMService>>()));

没有模式下载“,没有GGUF转换@,没有兼容性检查”“.这真的非常棒, 当你尝试不同的模型来观察什么最有效时 ”$.

“3. 十字路口”“-”平台

# Works on Mac (no CUDA support)
dotnet run  # Just works!

# Works on Linux ARM (Raspberry Pi?)
dotnet run  # Just works!

# Works in Codespaces/Gitpod
dotnet run  # Just works!

本地方法为 Windows @ ++NVIDIA 只@.+#

*4. 缩放性

// Handle 100 concurrent users? Easy with APIs
await Task.WhenAll(users.Select(u =>
    rag.GenerateSuggestionAsync(u.Draft)));

// Local? Limited by your single GPU

“5.”简单部署

# Deploy to Azure/AWS/GCP
dotnet publish -c Release
# Upload single binary, set env vars, done

# Local? Need to:
# - Include 12GB model files
# - Install CUDA on target machine
# - Ensure GPU drivers
# - Manage VRAM

贸易限制

*1. 隐私问题

您的博客内容将进入 OpenAI/AnthroopicMS K1

缓解@:

仅用于公共博客内容
检查提供者@' @%s 数据使用策略
OpenAI:非用于培训的 API 数据@ (as {2024)}
同样的承诺

如果您正在起草保密内容@' @,}(使用本地方法) @MS K2 @I'#我对此感到很满意,

2.网络依赖性

没有网络@=没有助手 @.

缓解@:

本地缓存先前的建议
执行脱机编辑模式
返回当地小型模型

时长

API调用“1-3”秒,

校验真实 :

本地端@: @ @ 0.5 @%s 一代
Cloud M: 2 一代
虽然这取决于你有多迫不及待

@4. 卖主锁定 #-#in

切换 API 时需要更改代码@. @%

缓解@:

// Use abstraction layer
public interface ILLMService
{
    // Switch providers easily
}

// Factory pattern
services.AddSingleton<ILLMService>(sp =>
{
    return config["Provider"] switch
    {
        "OpenAI" => new OpenAILLMService(...),
        "Anthropic" => new ClaudeLLMService(...),
        "Cohere" => new CohereLLMService(...),
        _ => throw new Exception("Unknown provider")
    };
});

两个世界最好的混合方法

您可混合本地和云端 ? @% 绝对的!

public class HybridEmbeddingService : IEmbeddingService
{
    private readonly LocalOnnxEmbedding _local;
    private readonly OpenAIEmbeddingService _cloud;
    private readonly bool _preferLocal;

    public async Task<float[]> GenerateEmbeddingAsync(string text)
    {
        if (_preferLocal && _local.IsAvailable())
        {
            return _local.GenerateEmbedding(text);  // Free, fast
        }

        return await _cloud.GenerateEmbeddingAsync(text);  // Fallback
    }
}

本地嵌入 @ (cheap, fast\ ), 云用于 LLM {(} 质量事件@):

本地端的嵌入@ : 保存 @ MS K1 月 {(tiny accept}\ , 承认=)
云端的LLM=: 获得 GPT+-4/claude 质量

这其实是我的 建议采用的方法@,#虽然I'我还在尝试看它是否正确平衡 #:#

本地端运行小型嵌入模型 @ (no GPU 需要@ MS K1
LLM 使用云层API
用于发展的本地Qdrant @,云用于生产

结论结论

“-”或至少“'”是迄今为止我的经验。

-=YTET -伊甸园字幕组=- 翻译: |---------|-------|-------| @| @设置时间 @ | @ @ I2-4 @ 小时 @ | @ @ @ 15 @ minutes @ MPK5 @ |#硬件要求 @|_NVIDIA GPU%|}任何电脑@|} @| 模范品质 @#| #7B#MS K3B @| GPT-4, Claude 3.5@| @|月费$@| $0}|}大概是 @$3-5 -=YTET -伊甸园字幕组=- 翻译: 本地的| 发送到APIs |交叉-平方形 @|视窗只MS K3Mac/Linux}/視窗@|# CUDA更新 |None MS K4

何时使用云端@ : @%

您没有 NVIDIA GPU 的 NVIDIA gPU
您在 Mac/Linux 上的 're
你想要最简单的道路
您的写作小于 @10 @ poss@/_ month
您信任有公共内容的云源提供者

何时使用本地端@ : @

你有 GPU 硬件
隐私是关键
您想要“ $0” 运行成本
您的写作超过 @20 @ possible@/_ month
你喜欢修补工匠

最佳办法#: RAG 架构完全相同 @-}你只要换掉LLM Service@!虽然我应该提醒 IZ'm仍然学习这里的“入门”

下一步步骤

试试今天下午设置云版
入列@: 处理您的博客文章
写入获取基于您过去内容的AI建议@ :
度量措施*:轨道成本和用途
决定@: 保持云彩或迁移到本地@?

RAG的美丽就是 您没有被锁定. 矢量数据库, buggingMS K2 和检索逻辑工作原理相同,无论您的 LLM 在您 GPU 上还是在 OpenAI}' s datacentre.

开始简单@, @ 证明值@ , @ 然后最优化 @ MS K2 @ I'# m 仍在学习什么最适合自己 @ I, @ 因此 I'}希望听到您如何使用 @ MPK6 @