# 为 Markdig 建立一个远程马克唐抓取器

<!--category-- Markdown, AI-Article,  MarkDig, ASP.NET Core, C#, API, Nuget, FetchExtension-->
<datetime class="hidden">2025-11-07T10:00</datetime>

# 一. 导言导言 导言 导言 导言

我在建立这个博客时所面临的一个挑战是如何有效地包括外部标记下的内容而不在任何地方手动复制和粘贴@. @%我想从我的 GitHub 库中获取 README 文件\,}{}包含来自其他项目的文件},并自动同步所有东西。

本文中#, @I'}将介绍我是如何建造的 `Mostlylucid.Markdig.FetchExtension` - 获取并缓存远程标记内容的完整解决方案, 支持多个存储后端@, 自动投票 @, 和一个 stale- while-revalidate chaching stand_ MS K5}

> NotezMSKO}这仍是前期释放 但我想把它弄出去 玩得开心点 但是可能还没成功

> 这篇文章是 AI 使用 cluude 代码生成的“- ” , 也帮助我建立这个功能 “. ”

> **最新更新***:添加 `disable="true"` 参数,所以我们现在可以正确演示标记而不被处理@! @%

> **最新消息: (+Nov+7,+2025)+**: 添加的内容表 @(TOC}) 生成特性@!使用 `[TOC]` 在您标记后自动从文档标题中生成可点击的目录@.}

看看这个源头 [在吉特胡卜山上为这个站点](https://github.com/scottgal/mostlylucidweb/tree/main/Mostlylucid.Markdig.FetchExtension).

[![Nuget 元数](https://img.shields.io/nuget/v/mostlylucid.Markdig.FetchExtension.svg)](https://www.nuget.org/packages/mostlylucid.Markdig.FetchExtension)
[![Nuget 元数](https://img.shields.io/nuget/dt/mostlylucid.Markdig.FetchExtension.svg)](https://www.nuget.org/packages/mostlylucid.Markdig.FetchExtension)

[TOC]

# 为何要建构这个#?

在潜入技术细节前 请让我解释一下问题 我有几个需要包括 外部标记内容的假想

1. **软件包 REEADME**: 当我写一个 Nuget 套件 I ' @ 已经发布@ MS K2} 我想直接加入 GitHub 的 README
2. **API 文件**经常改变的外部API docs 需要保持同步
3. **共享内容**: 生活在一个存储库中但需要出现在多个地方的文件
4. **业绩表现 实绩**: @ I don't want to fick this confession 在每页加载的“-}”页面上获取这个内容, 它会慢而浪费。

最天真的做法是使用 HTTP 客户端在需要时获取标记@ . @ but that's problemsQ:}但是这有问题吗?

- 每个请求都点击远程服务器
- 网络延迟影响页面负载时间
- 没有离线支持
- 没有处理短暂故障

我需要一些更聪明的@ : @ ficking once *,_ 智能缓存*\ MS K2} 自动刷新@ MPK3 并处理失败 优雅地 @ .}

# 建筑结构概览

扩展使用预处理方法, 而不是作为 Markdig 剖析管道中的一部分@.}这至关重要, 因为这意味着通过您的整个 Markdi 管道获取内容流\, 获得您所有的自定义延伸@ ,_ 语法强调 @ MS K3 和打字Q.}

```mermaid
graph TD
    A[Markdown with fetch tags] --> B[MarkdownFetchPreprocessor]
    B --> C{Check Cache}
    C -->|Fresh| D[Return Cached Content]
    C -->|Stale/Missing| E[Fetch from Remote URL]
    E -->|Success| F[Update Cache]
    E -->|Failure| G{Has Cached?}
    G -->|Yes| H[Return Stale Cache]
    G -->|No| I[Return Error Comment]
    F --> J[Replace fetch with Content]
    D --> J
    H --> J
    I --> J
    J --> K[Processed Markdown]
    K --> L[Your Markdig Pipeline]
    L --> M[Final HTML]

```

这里的关键见解是 **预处理前**马克迪输油管开始倒计时前 ,

1. 扫描 `<fetch>` 标签标记
2. 解析来自缓存或远程@) @% 1
3. 将标签替换为实际标记
4. 然后让Markdig 一起处理一切

这确保了一致性 *% -* 所有标记都得到相同的待遇 不管来源如何* .* 

# 基本语法

使用扩展名简单@ . @ 在您的标记上下Q:}#

```markdown
# My Documentation

<!-- Failed to fetch content from https://raw.githubusercontent.com/user/repo/main/README.md: HTTP 404: Not Found -->
```

扩展号是:

- 从 GitHub 获取 README
- 赶在#24+1小时前
- 返回后请求的缓存内容
- 当缓存过期后自动-rerefreew

# 存储提供者建筑

我遵循的设计原理之一是 **灵活度**不同的应用程序都有不同的需求 @.}一个小的演示应用程序 does\' 200)}t need PostgreSQL,}但多功能-}服务器的生产部署是#.}所以我建造了一个插接存储结构_:}

```mermaid
graph LR
    A[IMarkdownFetchService Interface] --> B[InMemoryMarkdownFetchService]
    A --> C[FileBasedMarkdownFetchService]
    A --> D[PostgresMarkdownFetchService]
    A --> E[SqliteMarkdownFetchService]
    A --> F[SqlServerMarkdownFetchService]
    A --> G[YourCustomService]

    B --> H[ConcurrentDictionary]
    C --> I[File System + SemaphoreSlim]
    D --> J[PostgreSQL Database]
    E --> K[SQLite Database]
    F --> L[SQL Server Database]
    G --> M[Your Storage Backend]

```

## 核心界面

每件东西都执行 `IMarkdownFetchService`:

```csharp
public interface IMarkdownFetchService
{
    Task<MarkdownFetchResult> FetchMarkdownAsync(
        string url,
        int pollFrequencyHours,
        int blogPostId = 0);

    Task<bool> RemoveCachedMarkdownAsync(
        string url,
        int blogPostId = 0);
}
```

简单且清洁的@.}每个执行程序都自行存储自己的方式 @,}但是接口仍然一致#.

## In -Memory memory stready:完美于演示

最简单的执行用途 `ConcurrentDictionary`:

```csharp
public class InMemoryMarkdownFetchService : IMarkdownFetchService
{
    private readonly ConcurrentDictionary<string, CacheEntry> _cache = new();
    private readonly IHttpClientFactory _httpClientFactory;
    private readonly ILogger<InMemoryMarkdownFetchService> _logger;

    public async Task<MarkdownFetchResult> FetchMarkdownAsync(
        string url,
        int pollFrequencyHours,
        int blogPostId)
    {
        var cacheKey = GetCacheKey(url, blogPostId);

        // Check cache
        if (_cache.TryGetValue(cacheKey, out var cached))
        {
            var age = DateTimeOffset.UtcNow - cached.FetchedAt;
            if (age.TotalHours < pollFrequencyHours)
            {
                _logger.LogDebug("Returning cached content for {Url}", url);
                return new MarkdownFetchResult
                {
                    Success = true,
                    Content = cached.Content
                };
            }
        }

        // Fetch fresh content
        var fetchResult = await FetchFromUrlAsync(url);

        if (fetchResult.Success)
        {
            _cache[cacheKey] = new CacheEntry
            {
                Content = fetchResult.Content,
                FetchedAt = DateTimeOffset.UtcNow
            };
        }
        else if (cached != null)
        {
            // Fetch failed, return stale cache
            _logger.LogWarning("Fetch failed, returning stale cache for {Url}", url);
            return new MarkdownFetchResult
            {
                Success = true,
                Content = cached.Content
            };
        }

        return fetchResult;
    }

    private static string GetCacheKey(string url, int blogPostId)
        => $"{url}_{blogPostId}";
}
```

如您所看到的, 以下的“:” 效果如下:

1. 从 URL 和博客 post ID 创建缓存密钥
2. 检查我们是否存有缓存内容,以及它是否是新鲜的@' @%s
3. 如果缓存是清新@ , @ {} 立即返回它
4. 尝试获取新内容
5. 成功时@, 更新缓存
6. 与缓存内容失败时 @,返回 stale 缓存@ (}%stale- while{-revalidate_!)
7. 没有缓存% , 返回错误时失败

此图案@ - @% 1 **staale@ - @ while @ MS K1 @ revalidate** @-对可靠性至关重要@. 即使GitHub 正在下降@MS K2你的網站繼續使用隱藏的內容@.

## 檔案 - @ based demment@ MS K1 @ 簡單持久性

对于单-Server部署}MS K1 file} -基封存工作非常成功 □:

```csharp
public class FileBasedMarkdownFetchService : IMarkdownFetchService
{
    private readonly string _cacheDirectory;
    private readonly IHttpClientFactory _httpClientFactory;
    private readonly ILogger<FileBasedMarkdownFetchService> _logger;
    private readonly SemaphoreSlim _fileLock = new(1, 1);

    public async Task<MarkdownFetchResult> FetchMarkdownAsync(
        string url,
        int pollFrequencyHours,
        int blogPostId)
    {
        var cacheKey = ComputeCacheKey(url, blogPostId);
        var cacheFile = GetCacheFilePath(cacheKey);

        await _fileLock.WaitAsync();
        try
        {
            // Check if file exists and is fresh
            if (File.Exists(cacheFile))
            {
                var fileInfo = new FileInfo(cacheFile);
                var age = DateTimeOffset.UtcNow - fileInfo.LastWriteTimeUtc;

                if (age.TotalHours < pollFrequencyHours)
                {
                    var cached = await File.ReadAllTextAsync(cacheFile);
                    return new MarkdownFetchResult
                    {
                        Success = true,
                        Content = cached
                    };
                }
            }

            // Fetch fresh
            var fetchResult = await FetchFromUrlAsync(url);

            if (fetchResult.Success)
            {
                await File.WriteAllTextAsync(cacheFile, fetchResult.Content);
            }
            else if (File.Exists(cacheFile))
            {
                // Return stale on fetch failure
                var stale = await File.ReadAllTextAsync(cacheFile);
                return new MarkdownFetchResult
                {
                    Success = true,
                    Content = stale
                };
            }

            return fetchResult;
        }
        finally
        {
            _fileLock.Release();
        }
    }

    private string GetCacheFilePath(string cacheKey)
        => Path.Combine(_cacheDirectory, $"{cacheKey}.md");

    private static string ComputeCacheKey(string url, int blogPostId)
    {
        var combined = $"{url}_{blogPostId}";
        using var sha256 = SHA256.Create();
        var bytes = Encoding.UTF8.GetBytes(combined);
        var hash = sha256.ComputeHash(bytes);
        return Convert.ToHexString(hash);
    }
}
```

关键点% : @% 1

1. 用途 `SemaphoreSlim` 用于线索@-}安全文件访问
2. 为创建安全文件名, Hashes the URL *+博客发布 ID 以创建安全的文件名
3. 使用文件修改时间确定新鲜度
4. 在整个应用程序重新启动中保持永久状态
5. 相同的标准@ - @ while @ MS K1 @ revalidate status

## 数据库存储@:production @-ready

您想要共享缓存@ . @ that'}在数据库提供商到 <% MS K5 > 的地方

```csharp
public class PostgresMarkdownFetchService : IMarkdownFetchService
{
    private readonly MarkdownCacheDbContext _dbContext;
    private readonly IHttpClientFactory _httpClientFactory;
    private readonly ILogger<PostgresMarkdownFetchService> _logger;

    public async Task<MarkdownFetchResult> FetchMarkdownAsync(
        string url,
        int pollFrequencyHours,
        int blogPostId)
    {
        var cacheKey = GetCacheKey(url, blogPostId);

        // Query cache
        var cached = await _dbContext.MarkdownCache
            .FirstOrDefaultAsync(c => c.CacheKey == cacheKey);

        if (cached != null)
        {
            var age = DateTimeOffset.UtcNow - cached.LastFetchedAt;
            if (age.TotalHours < pollFrequencyHours)
            {
                return new MarkdownFetchResult
                {
                    Success = true,
                    Content = cached.Content
                };
            }
        }

        // Fetch fresh
        var fetchResult = await FetchFromUrlAsync(url);

        if (fetchResult.Success)
        {
            if (cached == null)
            {
                cached = new MarkdownCacheEntry
                {
                    CacheKey = cacheKey,
                    Url = url,
                    BlogPostId = blogPostId
                };
                _dbContext.MarkdownCache.Add(cached);
            }

            cached.Content = fetchResult.Content;
            cached.LastFetchedAt = DateTimeOffset.UtcNow;
            await _dbContext.SaveChangesAsync();
        }
        else if (cached != null)
        {
            // Return stale
            return new MarkdownFetchResult
            {
                Success = true,
                Content = cached.Content
            };
        }

        return fetchResult;
    }
}
```

数据库模式很简单 @: @

```sql
CREATE TABLE markdown_cache (
    id SERIAL PRIMARY KEY,
    cache_key VARCHAR(128) NOT NULL UNIQUE,
    url VARCHAR(2048) NOT NULL,
    blog_post_id INTEGER NOT NULL,
    content TEXT NOT NULL,
    last_fetched_at TIMESTAMP WITH TIME ZONE NOT NULL,
    CONSTRAINT ix_markdown_cache_cache_key UNIQUE (cache_key)
);

CREATE INDEX ix_markdown_cache_url_blog_post_id
    ON markdown_cache(url, blog_post_id);
```

在多@ - @ server display *\ MS K1} 这让您可以缓存所有情况下的一致性 * _:

```mermaid
graph TB
    subgraph "Load Balancer"
        LB[Load Balancer]
    end

    subgraph "Application Servers"
        A1[App Server 1<br/>FetchExtension]
        A2[App Server 2<br/>FetchExtension]
        A3[App Server 3<br/>FetchExtension]
    end

    subgraph "Shared Cache"
        PG[(PostgreSQL<br/>markdown_cache table)]
    end

    subgraph "External Content"
        R1[Remote URL 1]
        R2[Remote URL 2]
        R3[Remote URL 3]
    end

    LB --> A1
    LB --> A2
    LB --> A3

    A1 <-->|Read/Write Cache| PG
    A2 <-->|Read/Write Cache| PG
    A3 <-->|Read/Write Cache| PG

    A1 -.->|Fetch if cache miss| R1
    A2 -.->|Fetch if cache miss| R2
    A3 -.->|Fetch if cache miss| R3

```

所有服务器都共享相同的缓存@ . 当服务器 @ MS K1 获取一个 README @ MOSK2 服务器\ 2 和\ MPK4 立即从该缓存内容 < MSKO5} 中受益时

# 设置扩展

启动时是直截了当 @ . @ first @ MPK1} 安装基套件@ MS K2

```bash
dotnet add package mostlylucid.Markdig.FetchExtension
```

然后选择您的存储提供者@: @%

```bash
# For in-memory (demos/testing)
# Already included in base package

# For file-based storage
# Already included in base package

# For PostgreSQL
dotnet add package mostlylucid.Markdig.FetchExtension.Postgres

# For SQLite
dotnet add package mostlylucid.Markdig.FetchExtension.Sqlite

# For SQL Server
dotnet add package mostlylucid.Markdig.FetchExtension.SqlServer
```

## ASP.NET 核心中的配置

在你的 `Program.cs`:

```csharp
using Mostlylucid.Markdig.FetchExtension;

var builder = WebApplication.CreateBuilder(args);

// Option 1: In-Memory (simplest)
builder.Services.AddInMemoryMarkdownFetch();

// Option 2: File-Based (persists across restarts)
builder.Services.AddFileBasedMarkdownFetch("./markdown-cache");

// Option 3: PostgreSQL (multi-server)
builder.Services.AddPostgresMarkdownFetch(
    builder.Configuration.GetConnectionString("MarkdownCache"));

// Option 4: SQLite (single server with DB)
builder.Services.AddSqliteMarkdownFetch("Data Source=markdown-cache.db");

// Option 5: SQL Server (enterprise)
builder.Services.AddSqlServerMarkdownFetch(
    builder.Configuration.GetConnectionString("MarkdownCache"));

var app = builder.Build();

// If using database storage, ensure schema exists
if (app.Environment.IsDevelopment())
{
    app.Services.EnsureMarkdownCacheDatabase();
}

// Configure the extension with your service provider
FetchMarkdownExtension.ConfigureServiceProvider(app.Services);

app.Run();
```

## 与您的标记下错合并

关键是预处理步骤 @._这里#'_%s 我如何将它融入我的博客@:}

```csharp
public class MarkdownRenderingService
{
    private readonly IServiceProvider _serviceProvider;
    private readonly MarkdownFetchPreprocessor _preprocessor;
    private readonly MarkdownPipeline _pipeline;

    public MarkdownRenderingService(IServiceProvider serviceProvider)
    {
        _serviceProvider = serviceProvider;
        _preprocessor = new MarkdownFetchPreprocessor(serviceProvider);

        _pipeline = new MarkdownPipelineBuilder()
            .UseAdvancedExtensions()
            .UseSyntaxHighlighting()
            .UseToc()  // Add TOC support for [TOC] markers
            .UseYourCustomExtensions()
            .Build();
    }

    public string RenderMarkdown(string markdown)
    {
        // Step 1: Preprocess to handle fetch tags
        var processed = _preprocessor.Preprocess(markdown);

        // Step 2: Run through your normal Markdig pipeline
        return Markdown.ToHtml(processed, _pipeline);
    }
}
```

流动是:

1. 您的标记包含 `<fetch>` 标签标记
2. 预处理器将它们溶解到实际标记
3. 合并计分通过马克迪
4. 所有东西都有您的自定义扩展@ , @ sttyling @ MS K1 @ etcs @ MPK2}

# 高级特征

## 内容生成表

软件包现在包含 **单独** 内容表 @( @TOC@)}扩展号!}(虽然它'}) 和抓取扩展包包一起包装了,} it{'}它们完全独立,可以自行使用。 您可以自动从您的文档中生成可点击的目录 <%'> 标题MS K8}

**基本用法 @:**

简单添加 `[TOC]` 标记中的任意位置@: @%

```markdown
# My Document

[TOC]

# Introduction
Content here...

# Getting Started
More content...

## Installation
Details...
```

这将生成所有标题的嵌套列表, 带有锁定链接@ : @ {}

```html
<nav class="ml_toc" aria-label="Table of Contents">
  <ul>
    <li><a href="#introduction">Introduction</a></li>
    <li><a href="#getting-started">Getting Started</a>
      <ul>
        <li><a href="#installation">Installation</a></li>
      </ul>
    </li>
  </ul>
</nav>
```

**自定义 CSS 类**

您可以指定用于发型的自定义 CSS 类@ : @

```markdown
[TOC cssclass="my-custom-toc"]
```

这将与您自定义的类创建@ : @

```html
<nav class="my-custom-toc" aria-label="Table of Contents">
  <!-- TOC content -->
</nav>
```

**它如何运作 :**

1. **自动-检测**: TOC 自动检测文档中的最低标题级别, 并相应调整@. 如果您的文档以 H2,开头, TOC 将 H2 视为顶级 @.

2. **ID代人***% : 标题自动被给予锁定连接的标识*\ :
   
   - @"# 启动@"} @ → @ `id="getting-started"`
   - "API参考 MS K1→ `id="api-reference"`

3. **括内结构**造物主建造了一个合适的巢穴 `ul/li` 显示文档等级结构的结构@.}{}

**启用 TOC 支持@: @%**

当配置您的 Markdig 管道时 @ , @ 添加 TOC 扩展@ MS K1\ {

```csharp
var pipeline = new MarkdownPipelineBuilder()
    .UseAdvancedExtensions()
    .UseToc()  // Add TOC support - position in pipeline doesn't matter!
    .Use<YourOtherExtensions>()
    .Build();
```

**管道定位@: @%** 不同于某些 Markdig 扩展@, @ TOC 扩展 **{\fn黑体\fs22\bord1\shad0\3aHBE\4aH00\fscx67\fscy66\2cHFFFFFF\3cH808080}不管你添加到哪里** 在编审中 @ . 扩展自动\ :

- 在分析器列表的开头插入其解析器 (}position {0)}\
- 在所有解析完成后重命名@ , @ {} 从整个文档中收集标题

所以你可以添加 `.UseToc()` @ - @ 开始@ MS K1 @ 中间 @ OMK2 @ 或管道配置的结束@ I.}

**重要的:** TOC 扩展名完全独立于抓取延伸号@. @ they'}他们刚为方便而一起包装的附加号@ ._ 您可以\ :} @

- 不经获取使用 TOC
- 使用不带 TOC 的抓取
- 两者同时使用

**注::** 如果从 GitHub 中获取 README 包含 `[TOC]`, 它将自动从此文档中生成目录@'}标题 @( 假设您已添加 `.UseToc()` 您的管道@ ). @%

## 链接转换

当获取远程标记@ ( @ 特别是来自 GitHub),} 相对链接断开\ MS K2} 此扩展可自动重写@ MPK3}

```markdown
<!-- Failed to fetch content from https://raw.githubusercontent.com/user/repo/main/docs/README.md: HTTP 404: Not Found -->
```

此变换@ : @

- `./CONTRIBUTING.md` → `https://github.com/user/repo/blob/main/docs/CONTRIBUTING.md`
- `../images/logo.png` → `https://github.com/user/repo/blob/main/images/logo.png`
- 保存绝对的 URL和锁定

执行时使用 Markdig AST 重写链接@: @%

```csharp
public class MarkdownLinkRewriter
{
    public static string RewriteLinks(string markdown, string sourceUrl)
    {
        var document = Markdown.Parse(markdown);
        var baseUri = GetBaseUri(sourceUrl);

        foreach (var link in document.Descendants<LinkInline>())
        {
            if (IsRelativeLink(link.Url))
            {
                link.Url = ResolveRelativeLink(baseUri, link.Url);
            }
        }

        using var writer = new StringWriter();
        var renderer = new NormalizeRenderer(writer);
        renderer.Render(document);
        return writer.ToString();
    }

    private static bool IsRelativeLink(string url)
    {
        if (string.IsNullOrEmpty(url)) return false;
        if (url.StartsWith("http://") || url.StartsWith("https://")) return false;
        if (url.StartsWith("#")) return false;  // Anchor
        if (url.StartsWith("mailto:")) return false;
        return true;
    }
}
```

## 获取元数据摘要

您可以在上次获取内容时显示读者@ : @

```markdown
<!-- Failed to fetch content from https://api.example.com/status.md: HTTP request failed: Name or service not known (api.example.com:443) -->
```

使用页脚@ : @

> _内容从 [https @://api}.example}MS2com/}状态=.md](https://api.example.com/status.md) 校对:Portnoy_

或自定义模板@: @%

```markdown
<!-- Failed to fetch content from https://example.com/docs.md: HTTP 404: Not Found -->
```

产出=:~~~~

> 最新更新的@: @ @06_ 1月 @MS K2} @14:30Q}状态=|#Satus :}缓存 *|}下次刷新 #:}在 @MSKO8#小时

可用的占位符@: @%

- `{retrieved:format}` - 最后一次取回日期 @/_time
- `{age}` @ - @ human@ MS K1_ 自抓取后可读取的时间@ info
- `{url}` -来源URL
- `{nextrefresh:format}` - 内容何时会刷新
- `{pollfrequency}` -缓存时间(小时)
- `{status}` - 缓存状态 @_resh}/#cached/{stale}MS K4}

## 禁用文件处理

写入获取扩展名的文档时“% (” 如此文章 @ MS K1} 您需要一种方法来显示标签而不被处理@.使用 `disable="true"` 属性@: @%

```markdown
<!-- This will be processed and fetch content -->
<!-- Failed to fetch content from https://example.com/README.md: HTTP 404: Not Found -->

<!-- This will NOT be processed - useful for documentation -->
<!-- Failed to fetch content from https://example.com/README.md: HTTP 404: Not Found -->
```

禁用标签仍保留在标记下方为 @ - @ is@ MS K1# 完美 。

- 写入关于扩展名本身的文档
- 在教义中创建示例
- 显示标签语法而不触发抓取

这对两者都管用 `<fetch>` 和 `<fetch-summary>` 标签@: @%

```markdown
<!-- No fetch data available for https://example.com/api/status.md -->
```

## 监测事件系统

扩展名发布所有抓取操作的事件@: @%

```csharp
public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddPostgresMarkdownFetch(connectionString);

        var sp = services.BuildServiceProvider();
        var eventPublisher = sp.GetRequiredService<IMarkdownFetchEventPublisher>();

        // Subscribe to events
        eventPublisher.FetchBeginning += (sender, args) =>
        {
            Console.WriteLine($"Fetching {args.Url}...");
        };

        eventPublisher.FetchCompleted += (sender, args) =>
        {
            var source = args.WasCached ? "cache" : "remote";
            Console.WriteLine($"Fetched {args.Url} from {source} in {args.Duration.TotalMilliseconds}ms");
        };

        eventPublisher.FetchFailed += (sender, args) =>
        {
            Console.WriteLine($"Failed to fetch {args.Url}: {args.ErrorMessage}");
        };
    }
}
```

这使得您更容易与应用程序 Insights @,Prometheus@,或您的伐木基础设施整合=:

```mermaid
sequenceDiagram
    participant MD as Markdown Processor
    participant EP as Event Publisher
    participant FS as Fetch Service
    participant ST as Storage Backend
    participant L as Your Listeners

    MD->>EP: FetchBeginning
    EP->>L: Notify FetchBeginning
    EP->>FS: FetchMarkdownAsync(url)
    FS->>ST: Check Cache
    alt Cache Fresh
        ST-->>FS: Cached Content
        FS->>EP: FetchCompleted (cached=true)
    else Cache Stale/Missing
        FS->>FS: HTTP GET
        alt Success
            FS->>ST: Update Cache
            ST-->>FS: OK
            FS->>EP: FetchCompleted (cached=false)
        else Failure
            FS->>EP: FetchFailed
            EP->>L: Notify FetchFailed
        end
    end
    EP->>L: Notify FetchCompleted
    EP->>L: Notify ContentUpdated
    FS-->>MD: MarkdownFetchResult

    Note over L: Listeners can be: Logging, Metrics, Telemetry, Webhooks
```

# 详细缓存战略

缓冲行为遵循州立机器模式

```mermaid
stateDiagram-v2
    [*] --> CheckCache: Fetch Request

    CheckCache --> Fresh: Cache exists & age < pollFrequency
    CheckCache --> Stale: Cache exists & age >= pollFrequency
    CheckCache --> Missing: No cache entry

    Fresh --> ReturnCached: Return cached content
    ReturnCached --> [*]

    Stale --> FetchRemote: Attempt HTTP GET
    Missing --> FetchRemote: Attempt HTTP GET

    FetchRemote --> UpdateCache: Success
    FetchRemote --> HasStale: Failure

    UpdateCache --> ReturnFresh: Return new content
    ReturnFresh --> [*]

    HasStale --> ReturnStale: Return stale cache
    HasStale --> ReturnError: No cache available

    ReturnStale --> [*]
    ReturnError --> [*]

    note right of Fresh
        pollFrequency = 0
        means always stale
    end note

    note right of HasStale
        Stale-while-revalidate
        pattern ensures uptime
    end note
```

这里的关键洞察力 @ : @

1. **新鲜缓存新快取** {\fn华文楷体\fs16\1cHE0E0E0}没有网络被击中
2. **存储缓存** - 尝试获取 fresh\ MS K1 但如果抓取失败, 返回到停滞状态@ info: tooltip
3. **缺少缓存** - 必须获取或返回错误
4. **零彩票频率** “-” 总是会获取新鲜的“(” 用于测试“MS K2”

此模式被称为 **staale@ - @ while @ MS K1 @ revalidate** 即使你的源头下降 , 你的网站继续提供缓存内容 @.}

# CA缓存清除和管理

有时您需要手动失效缓存@ : @

```csharp
public class MarkdownController : Controller
{
    private readonly IMarkdownFetchService _fetchService;

    public async Task<IActionResult> InvalidateCache(string url)
    {
        var removed = await _fetchService.RemoveCachedMarkdownAsync(url);

        if (removed)
        {
            return Ok(new { message = "Cache invalidated" });
        }

        return NotFound(new { message = "No cache entry found" });
    }
}
```

内容更改时通过 Webhooks 访问@: @

```csharp
// GitHub webhook notifies of README update
app.MapPost("/webhooks/github", async (
    GitHubWebhookPayload payload,
    IMarkdownFetchService fetchService) =>
{
    if (payload.Repository?.FullName == "user/repo" &&
        payload.Commits?.Any(c => c.Modified?.Contains("README.md") == true) == true)
    {
        var url = "https://raw.githubusercontent.com/user/repo/main/README.md";
        await fetchService.RemoveCachedMarkdownAsync(url);

        return Results.Ok(new { message = "Cache invalidated" });
    }

    return Results.Ok(new { message = "No action needed" });
});
```

# 延长期测试

扩展包括全面测试@.这里 @'}我如何构建它们_:

```csharp
public class MarkdownFetchServiceTests
{
    [Fact]
    public async Task FetchMarkdownAsync_CachesContent()
    {
        // Arrange
        var services = new ServiceCollection();
        services.AddLogging();
        services.AddInMemoryMarkdownFetch();
        var sp = services.BuildServiceProvider();

        var fetchService = sp.GetRequiredService<IMarkdownFetchService>();
        var url = "https://raw.githubusercontent.com/user/repo/main/README.md";

        // Act - First fetch (from network)
        var result1 = await fetchService.FetchMarkdownAsync(url, 24, 0);

        // Act - Second fetch (from cache)
        var result2 = await fetchService.FetchMarkdownAsync(url, 24, 0);

        // Assert
        Assert.True(result1.Success);
        Assert.True(result2.Success);
        Assert.Equal(result1.Content, result2.Content);
    }

    [Fact]
    public async Task FetchMarkdownAsync_ReturnsStaleOnFailure()
    {
        // Arrange
        var services = new ServiceCollection();
        services.AddLogging();
        services.AddInMemoryMarkdownFetch();
        var sp = services.BuildServiceProvider();

        var fetchService = sp.GetRequiredService<IMarkdownFetchService>();
        var url = "https://httpstat.us/200?sleep=100";

        // Act - First fetch succeeds
        var result1 = await fetchService.FetchMarkdownAsync(url, 0, 0);

        // Change URL to fail
        var badUrl = "https://httpstat.us/500";

        // Act - Second fetch fails, should return stale
        var result2 = await fetchService.FetchMarkdownAsync(badUrl, 0, 0);

        // Assert
        Assert.True(result1.Success);
        // Even though fetch failed, we return success with stale content
        Assert.True(result2.Success);
    }
}
```

# 业绩考量

扩展名是为性能设计的 @:

1. **同时词典** -in-ime 实施用途 `ConcurrentDictionary` 用于线索@-}安全访问
2. **夏马磷** -File @-基使用同步锁定以防止种族条件
3. **数据库指数** -所有数据库提供者都有关于缓存密钥的适当索引
4. **HTTP 客户共享** -用途 `IHttpClientFactory` 高效连接再利用
5. **Async 一路前进** - @ no black calls * 所有东西都是灰色的

我家里服务器上的典型性能数字 @:

- Cache hit: < _1ms
- 在远程服务器上的网络@: @ 200-500}%s @ I(# dedepends 中丢失缓存@ MS K3 {
- 数据库搜索@: @ @ 2-5_%s @ I(PostgreSQL, 并配有正确的索引@ MS K3}
- 文件系统搜索@ : @ MS K1}%s

# 与吉特胡布行动部署

我用 OIDC 认证@: 来使用 GitHub Action 发布 Nuget 的所有四个软件包

```yaml
name: Publish Markdig.FetchExtension

on:
  push:
    tags:
      - 'fetchextension-v*.*.*'

permissions:
  id-token: write
  contents: read

jobs:
  build-and-publish:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Setup .NET
      uses: actions/setup-dotnet@v4
      with:
        dotnet-version: '9.0.x'

    - name: Extract version from tag
      id: get_version
      run: |
        TAG=${GITHUB_REF#refs/tags/fetchextension-v}
        echo "VERSION=$TAG" >> $GITHUB_OUTPUT

    - name: Build
      run: dotnet build Mostlylucid.Markdig.FetchExtension/Mostlylucid.Markdig.FetchExtension.csproj --configuration Release -p:Version=${{ steps.get_version.outputs.VERSION }}

    - name: Pack
      run: dotnet pack Mostlylucid.Markdig.FetchExtension/Mostlylucid.Markdig.FetchExtension.csproj --configuration Release --no-build -p:PackageVersion=${{ steps.get_version.outputs.VERSION }} --output ./artifacts

    - name: Login to NuGet (OIDC)
      id: nuget_login
      uses: NuGet/login@v1
      with:
        user: 'mostlylucid'

    - name: Publish to NuGet
      run: dotnet nuget push ./artifacts/*.nupkg --api-key ${{ steps.nuget_login.outputs.NUGET_API_KEY }} --source https://api.nuget.org/v3/index.json --skip-duplicate
```

OIDC 方法比存储 API 键“ - ” GitHub 生成短@- ” 更安全, 而非自动保存活标牌“ I. ”

# 世界使用量

@M'_@set:我如何在博客上使用它?

```markdown
# My NuGet Package Documentation

Here's the official README from GitHub:

# Umami.Net

**Version 1.1.0** · [Upgrade guide](https://github.com/scottgal/mostlylucidweb/blob/umamiv1.1.0/Umami.Net/MIGRATION.md) · [Release notes](https://github.com/scottgal/mostlylucidweb/blob/umamiv1.1.0/Umami.Net/RELEASE.md)

A .NET 8, 9 and 10 client for [Umami Analytics](https://umami.is), reviewed against
[Umami 3.4.0](https://github.com/umami-software/umami/releases/tag/v3.4.0).

The library sends page views, events, visitor identity and Web Vitals, and reads
website analytics. It supports self-hosted username/password or API-key
credentials, and the current Umami Cloud Bearer API-key protocol.

## Install and register

```sh
dotnet add package Umami.Net --version 1.1.0
```

```csharp
builder.Services.SetupUmamiClient(builder.Configuration); // Collection and background sender
builder.Services.SetupUmamiData(builder.Configuration);   // Authenticated analytics API
```

两个扩展使用 `Analytics` 配置区域 @.}注册或
注册并不改变您的应用程序@' @%s 日志水平或提供商 @.}

```json
{
  "Analytics": {
    "UmamiPath": "https://analytics.example.com/",
    "WebsiteId": "12345678-1234-1234-1234-123456789abc",
    "ApiVersion": "V3"
  }
}
```

通过用户秘密或环境变量而不是承诺来设置机密
{\fn黑体\fs22\bord1\shad0\3aHBE\4aH00\fscx67\fscy66\2cHFFFFFF\3cH808080}他们自己... `Analytics__ApiKey` 对账户 API 键
客户端发送 `Authorization: Bearer ...`
并且不调用这些密钥被屏蔽的指定端点@, @%

OpenóM,供应 `Analytics__Username` 和 `Analytics__Password`*. 客户
在\,}%s 中的日志用 POST 校验已有的标牌 `/api/auth/verify`, 重试 a
@401在强制重新登录后,
@ 401 @ 不带重复的永久@ MS K1} A {2} *FA 部分符号不被接受为 a
已完成对login @ ;}使用一个 API 密钥,用于未受人注意的访问 <% K1 fA MSSK2}启用的账户@.\ {}

Umami 云端=,}配置 `UmamiPath` 计为 `https://api.umami.is/v1/` (或a
区域终点,例如 `/v1/eu/`), `UseCloudApi: true`,和 `ApiKey`# .#
客户端保存该基础路径,并删除自@-}主机 `/api` 前缀@ . @%
收藏使用 tracker @ / recollect主机@ MS K1}而非已认证的云数据
如果您的端点不同, API; 单独注册收藏客户端@ .

API 键和现代分析路径需要一个支持它们的服务器@. @%
此软件包不升级您的 Umami 服务器@ , @ 迁移数据库或启用
服务器功能, 如会话重放或 MCP.

## 跟踪追踪

注射 `UmamiClient` 用于立即收藏@: @%

```csharp
using var response = await client.TrackPageView("/products?utm_source=newsletter", "Products");
response.EnsureSuccessStatusCode();

using var purchase = await client.Track("purchase", new UmamiEventData
{
    ["revenue"] = 49.95,
    ["currency"] = "GBP"
}, distinctId: "customer-123");
```

B. 返回的方法 `HttpResponseMessage` 将所有权转让到调用器@; @ 丢弃
回应 *.* `*AndDecode` 处理其反应和返回分析的方法
取代元数据@. @% `SendAsync` 添加取消支持而不更改
现有 `Send`, `Track`, `TrackPageView` 或 `Identify` 签名@.

当前收藏器返回 `{ cache, sessionId, visitId }`. 解码结果
曝光 `SessionId`, `VisitId` 和 `Cache`; `Id` 也包含当前届会
仍被解码为分析元数据=.}它们都是
未用作认证的证明@ . @% `{ "beep": "boop" }` 返回返回 `BotDetected`.

```csharp
var result = await client.TrackPageViewAndDecode("/landing");
using var next = await client.SendAsync(new UmamiPayload
{
    Url = "/pricing",
    CacheToken = result?.Cache,
    DistinctId = "customer-123",
    Tag = "campaign-a",
    Timestamp = DateTimeOffset.UtcNow.ToUnixTimeSeconds()
}, cancellationToken: cancellationToken);
```

缓存标记属于特定访问者@.}它们被明确传递到
有效载荷 `x-umami-cache`@;_BARBAR_客户从未在共享默认信头中存储过一个@.}#MSK1_
Umami @3.4识别访客使用 `DistinctId` (`id` {\fn黑体\fs22\bord1\shad0\3aHBE\4aH00\fscx67\fscy66\2cHFFFFFF\3cH808080}在线上... `SessionId`
在有效载荷上保留用于遗留客户端@;}当前服务器计算会话
而非接受调用者@- @%s 选中的会话 UUID_.}

明确事件数据@,IP 地址和用户{-}代理重叠有效载荷
客户端捕捉到入境用户代理@,}路径
基底@,}路径和查询字符串 @.}查詢字串保留 UTM 参数_.}在反向
ưμ㼯A
在收集活动之前 `RemoteIpAddress` 表示要访问的访客@.

`UseDefaultUserAgent` 是为现有整合保留的明确超值@;
正常收集保存真实数据
访问代理并允许 Umami' @%s bott 过滤应用@ .}

### 网络生命

```csharp
using var response = await client.TrackPerformanceAsync(new UmamiPayload
{
    Url = "/checkout", Title = "Checkout",
    Lcp = 1800, Inp = 120, Cls = 0.04, Fcp = 950, Ttfb = 180
});
```

时间字段为毫秒@ ; @ CLS 是无单位的分数 @ MS K1} 校验值
相对于当前收藏家@' @s range_. `Timestamp` 连结 **秒数秒**;
将分析请求日期转换为 Unix **毫秒**.

### 背景交付情况

注射 `UmamiBackgroundSender` 以捕捉请求元数据并输入送件列表@: @%

```csharp
await sender.TrackPageView("/products", "Products");
await sender.Track("checkout", new UmamiEventData { ["cartSize"] = 3 });
await sender.Identify(distinctId: "customer-123");
```

在 @ - memory 队列中, 维持在 < MS K1} 事件@ . Enqueue 操作等待全@ I, {
提供后压而不是无约束的内存增长@ . @% 关闭完成
在主机取消截止时间@, @ then
取消快件交付@ . @ 进程崩溃可能丢失队列数据@ MPK1\} 使用耐用发箱
无法接受的发送失败已登录, 没有停止阅读@. @%
追蹤資料寫入未自动重新查詢@, @% @ because a lost conview
否则,您可以双倍% - @ 计算事件@ MS K1 @ {

## 现有分析方法

注射 `UmamiDataService`:

```csharp
var request = new MetricsRequest
{
    StartAtDate = DateTime.UtcNow.AddDays(-7),
    EndAtDate = DateTime.UtcNow,
    Type = MetricType.path,
    Limit = 20,
    UtmSource = "newsletter"
};
var metrics = await data.GetMetrics(request);
if (metrics.Status == HttpStatusCode.OK)
    foreach (var row in metrics.Data ?? []) Console.WriteLine($"{row.x}: {row.y}");
```

现有方法仍然可用 @:} `GetStats`, `GetPageViews`, `GetMetrics`,
`GetExpandedMetrics`, `GetEventsSeries`,和 `GetActiveUsers`*.# 州政府正常化*
两者的遗产 `{ value, prev }` 和当前数字总计加 `comparison`
将现有 `.pageviews.value` / `.pageviews.prev` 活动访问者
接受遗产 `x` 和当前 `visitors` 字段@. @%

`ApiVersion` 默认默认值 `V3`. `MetricType.url` 地图到 `type=path` 在 V3.集
`V2` 以保留旧的公制值@;_% set `V1` 明确遗产
`path` → `url` 和 `hostname` → `host` 版本检测是
未自动@ . @ 无效的\ MS K1} 请不要静默重写, 重新提交@ MPK2 @ {
兼容性映射无法预示旧服务器上存在的所有功能@. @%

计量层面包括: `distinctId` 和 `utmSource`, `utmMedium`,
`utmCampaign`, `utmContent`, `utmTerm`@; 他们的骆驼细胞的电线名称被保存 #.#
`MetricType.event` 列表中的活动名称不需要事件过滤器@. @%
当未提供时区时, 支持分钟桶@; @ 事件序列默认为 UTC =% 1}
常见请求包括不同@-_sidentity @,}UTM,}事件=-}类型%,}匹配和弹出
过滤器@.过滤器是 URL @-encoded 一旦 ;}运算符和regex 逗号被保存 *%.}

## 当前网站分析

Umami @3.4 曝光网站@-scoped 获得报告计算端点\ :

```csharp
var report = await data.GetAnalyticsAsync(AnalyticsEndpoint.PerformanceChart,
    new AnalyticsRequest
    {
        StartAtDate = DateTime.UtcNow.AddDays(-7), EndAtDate = DateTime.UtcNow,
        Unit = Unit.day, Timezone = "Europe/London", Metric = "lcp"
    }, cancellationToken);
```

`AnalyticsEndpoint` 目标数= ,}热映射~,
, 保留@, 性能状态 stats/cchart/Qm4MS K4UTM 度量=,收入
Stats+/{zchart} /},}会话与事件按端点标准不同,使用
[正式合同](https://github.com/umami-software/umami/blob/v3.4.0/docs/report-api-migration.md).
例如,“,”收入需要 `Currency`, UTM 需要一个 `Type` 诸如
`utm_source`, 和漏斗预览需要 JSON `Steps` 加 a `Window`.

默认结果为 `UmamiResult<JsonElement>`@,_ 保存每个数据集@'%s
实际形状@. @% `GetAnalyticsAsync<T>` 支持调用@ - @ 定义的打字模型 @ MS K1
这些方法暴露了分析数据集@;}它们不会形成 Umami 界面或
记录浏览器会话重放流@. @%

### 说明说明说明

```csharp
var note = await data.CreateAnnotationAsync(new AnnotationInput(
    DateTimeOffset.UtcNow, "Website refresh deployed", AllDay: false), cancellationToken);
var notes = await data.GetAnnotationsAsync(page: 1, pageSize: 20, cancellationToken: cancellationToken);
// UpdateAnnotationAsync(id, input), GetAnnotationAsync(id), DeleteAnnotationAsync(id)
```

附加说明已键入输入@ , @ 响应和修正列表@ I._ 创建\ MS K2{ 更新中
DELETE,}与当前合同相符的DELEteMS K1}{.必须注意
包含“ 1–500” 字符@ MS K1} 这些方法只当被明确命名为“ MPK2 ” 时才执行写作 。 @ info

### 其他网站资源

`SendWebsiteRequestAsync<T>` 处理其他有文件记录的资源 @,}包括
已保存的漏斗@ / @ goals @ MS K1 @ section @ MPK2} 会话细节和重放元数据=.供应 a
它拒绝绝对的 URL=,} 横跨并嵌入
查询字符串. 网站 ID 和主机总是来自配置@.

```csharp
var goals = await data.SendWebsiteRequestAsync<JsonElement>(HttpMethod.Get, "goals",
    new Dictionary<string, string?> { ["page"] = "1", ["pageSize"] = "20" },
    cancellationToken: cancellationToken);
```

该方法保留结构化的 JSON 并允许输入结果模型@.资源
被配置的 Umami 服务器的 schemas 和权限仍保留在 schemas and periors 中@. @ it does
未披露账户 API-}管理或行政路线_.

对于助理访问 < ,\ > Umami *%3.4}提供自己的服务 [MCP 服务器](https://github.com/umami-software/umami/blob/v3.4.0/packages/mcp/README.md).
使用带有相应的 API 键的服务器@ ; @% 此软件包是一个 HTTP 分析器
I,}不是 MCP 执行@.

## 失败和注销

分析运输故障返回 `UmamiResult<T>`: MS K1 网络故障,
用于 HTTP 超时@, @\ 502对于无效的 JSON 响应 @,和服务器状态
HTTP 失败时
`OperationCanceledException`无效的本地参数和日期范围丢弃
在发送请求前@. @ 响应和失败的重试尝试被处理@ .

仅接收在 DI 注册中受约束的 Transient@/429 @ retription\ .
401 重新认证与一次重试 MS K1 中止请求是分开的
未为瞬态运输错误重新搜索@ . @ API-} key {401} 响应不会触发
密码后退@ . @ 日志省略 auth /} cache matters\ MS K2}有效载荷和查询字符串=.}

## 审定和释放

```sh
dotnet test Umami.Net.Test/Umami.Net.Test.csproj
scripts/test-umami-integration.sh # Spins up disposable Umami 3.4.0 + PostgreSQL; 51 real cases per target
dotnet build Umami.Net/Umami.Net.csproj -c Release
dotnet pack Umami.Net/Umami.Net.csproj -c Release --no-build -o /tmp/umami-pack
```

套件目标网@8.0,net @9.0}和释放标记的使用 `umamiv*.*.*`;
网站网页 `release-*` 标签不发布 NuGet 软件包@. @ 1 见
[调查结果的审评和核查](https://github.com/scottgal/mostlylucidweb/blob/main/Umami.Net/REVIEW.md) 用于当前更新的域@. @%

# 安装装置

软件包可在 Nuget ...\ 上查阅



This keeps documentation synchronized automatically. When I update the README on GitHub, the blog post stays in sync (within 24 hours).

# Conclusion

Building this extension taught me several things:

1. **Preprocessing > Parsing** - Handling fetch tags before Markdig ensures consistency
2. **Stale-While-Revalidate** - This pattern is incredibly valuable for reliability
3. **Pluggable Storage** - Different deployments need different solutions
4. **Events for Observability** - Being able to monitor fetch operations is crucial
5. **Thread Safety Matters** - Concurrent access needs careful handling
6. **Bonus Features** - The package now includes a separate TOC extension for generating tables of contents

The extension is open source and available on NuGet:

- [mostlylucid.Markdig.FetchExtension](https://www.nuget.org/packages/mostlylucid.Markdig.FetchExtension)
- [mostlylucid.Markdig.FetchExtension.Postgres](https://www.nuget.org/packages/mostlylucid.Markdig.FetchExtension.Postgres)
- [mostlylucid.Markdig.FetchExtension.Sqlite](https://www.nuget.org/packages/mostlylucid.Markdig.FetchExtension.Sqlite)
- [mostlylucid.Markdig.FetchExtension.SqlServer](https://www.nuget.org/packages/mostlylucid.Markdig.FetchExtension.SqlServer)

Source code is on GitHub: [scottgal/mostlylucidweb](https://github.com/scottgal/mostlylucidweb/tree/main/Mostlylucid.Markdig.FetchExtension)

If you're building a documentation site, blog, or any application that needs to include external markdown content, give it a try. It's designed to be simple to use but powerful enough for production deployments.