Yksi haasteita, joita jouduin kohtaamaan tätä blogia rakennettaessa, oli se, miten ulkoiset markown-sisällöt saadaan tehokkaasti mukaan kopioimatta ja liittämättä niitä käsin kaikkialle.
Halusin hakea README-tiedostoja GitHub-arkistoistani, sisältää asiakirjoja muista projekteista ja pitää kaiken automaattisesti synkronoituna.Mostlylucid.Markdig.FetchExtensionMikä on ratkaisu?
Räätälöity Markdig-laajennus, joka hakee etäleimauksen ja tallentaa sen älykkäästi.
Tässä viestissä kerron, miten rakensin
**- täydellinen ratkaisu kauko-ohjattavan maalintekosisällön hakemiseen ja välimuistiin useiden säilytystallenteiden, automaattisten äänestysten ja vanhentuneen välimuistikuvion avulla.**HUOMAUTUS: Tämä on vielä ennakkojulkaisu, mutta halusin saada sen julki.
disable="true"Pidä hauskaa, mutta se ei ehkä toimi vielä.
**Tämä artikkeli on tekoäly luotu - käyttäen laudan koodia, joka myös auttoi minua rakentamaan ominaisuuden.**PÄIVITYS
[TOC]: Lisätty
Parametri, jotta voimme nyt demota tunnisteet kunnolla ilman, että niitä käsitellään.
Katso lähde täältä
API-dokumentaatio
: En halua hakea tätä sisältöä jokaisesta sivukuormasta - se olisi hidasta ja tuhlailevaa
Mutta se on ongelmallista:
graph TD
A[Markdown with fetch tags] --> B[MarkdownFetchPreprocessor]
B --> C{Check Cache}
C -->|Fresh| D[Return Cached Content]
C -->|Stale/Missing| E[Fetch from Remote URL]
E -->|Success| F[Update Cache]
E -->|Failure| G{Has Cached?}
G -->|Yes| H[Return Stale Cache]
G -->|No| I[Return Error Comment]
F --> J[Replace fetch with Content]
D --> J
H --> J
I --> J
J --> K[Processed Markdown]
K --> L[Your Markdig Pipeline]
L --> M[Final HTML]
Jokainen pyyntö osuu etäpalvelimeenVerkon latenssi vaikuttaa sivujen latausaikoihinEi offline-tukea
<fetch>Tarvitsin jotain fiksumpaa: nouta kerran, välimuisti älykkäästi, virkistä automaattisesti ja käsittele epäonnistumiset sulavasti.Avainnäkemys tässä on
# My Documentation
<!-- Failed to fetch content from https://raw.githubusercontent.com/user/repo/main/README.md: HTTP 404: Not Found -->
Ennen kuin maali putoaa Markdigin putkeen, me:
Tämä takaa johdonmukaisuuden - kaikki markdownit saavat saman kohtelun lähteestään riippumatta.PerussyntaksiLaajennuksen käyttö on yksinkertaista.
graph LR
A[IMarkdownFetchService Interface] --> B[InMemoryMarkdownFetchService]
A --> C[FileBasedMarkdownFetchService]
A --> D[PostgresMarkdownFetchService]
A --> E[SqliteMarkdownFetchService]
A --> F[SqlServerMarkdownFetchService]
A --> G[YourCustomService]
B --> H[ConcurrentDictionary]
C --> I[File System + SemaphoreSlim]
D --> J[PostgreSQL Database]
E --> K[SQLite Database]
F --> L[SQL Server Database]
G --> M[Your Storage Backend]
Juuri noin!IMarkdownFetchService:
public interface IMarkdownFetchService
{
Task<MarkdownFetchResult> FetchMarkdownAsync(
string url,
int pollFrequencyHours,
int blogPostId = 0);
Task<bool> RemoveCachedMarkdownAsync(
string url,
int blogPostId = 0);
}
Laajennus tulee olemaan seuraava:
Viipaloi sitä 24 tuntiaConcurrentDictionary:
public class InMemoryMarkdownFetchService : IMarkdownFetchService
{
private readonly ConcurrentDictionary<string, CacheEntry> _cache = new();
private readonly IHttpClientFactory _httpClientFactory;
private readonly ILogger<InMemoryMarkdownFetchService> _logger;
public async Task<MarkdownFetchResult> FetchMarkdownAsync(
string url,
int pollFrequencyHours,
int blogPostId)
{
var cacheKey = GetCacheKey(url, blogPostId);
// Check cache
if (_cache.TryGetValue(cacheKey, out var cached))
{
var age = DateTimeOffset.UtcNow - cached.FetchedAt;
if (age.TotalHours < pollFrequencyHours)
{
_logger.LogDebug("Returning cached content for {Url}", url);
return new MarkdownFetchResult
{
Success = true,
Content = cached.Content
};
}
}
// Fetch fresh content
var fetchResult = await FetchFromUrlAsync(url);
if (fetchResult.Success)
{
_cache[cacheKey] = new CacheEntry
{
Content = fetchResult.Content,
FetchedAt = DateTimeOffset.UtcNow
};
}
else if (cached != null)
{
// Fetch failed, return stale cache
_logger.LogWarning("Fetch failed, returning stale cache for {Url}", url);
return new MarkdownFetchResult
{
Success = true,
Content = cached.Content
};
}
return fetchResult;
}
private static string GetCacheKey(string url, int blogPostId)
=> $"{url}_{blogPostId}";
}
Palauta välimuistiin tallennettua sisältöä seuraavissa pyynnöissä
Niinpä rakensin tukittavan säilytysarkkitehtuurin:YdinliitännätKaikki toimii
Jokainen toteutus käsittelee tallennuksen omalla tavallaan, mutta rajapinta pysyy johdonmukaisena.
public class FileBasedMarkdownFetchService : IMarkdownFetchService
{
private readonly string _cacheDirectory;
private readonly IHttpClientFactory _httpClientFactory;
private readonly ILogger<FileBasedMarkdownFetchService> _logger;
private readonly SemaphoreSlim _fileLock = new(1, 1);
public async Task<MarkdownFetchResult> FetchMarkdownAsync(
string url,
int pollFrequencyHours,
int blogPostId)
{
var cacheKey = ComputeCacheKey(url, blogPostId);
var cacheFile = GetCacheFilePath(cacheKey);
await _fileLock.WaitAsync();
try
{
// Check if file exists and is fresh
if (File.Exists(cacheFile))
{
var fileInfo = new FileInfo(cacheFile);
var age = DateTimeOffset.UtcNow - fileInfo.LastWriteTimeUtc;
if (age.TotalHours < pollFrequencyHours)
{
var cached = await File.ReadAllTextAsync(cacheFile);
return new MarkdownFetchResult
{
Success = true,
Content = cached
};
}
}
// Fetch fresh
var fetchResult = await FetchFromUrlAsync(url);
if (fetchResult.Success)
{
await File.WriteAllTextAsync(cacheFile, fetchResult.Content);
}
else if (File.Exists(cacheFile))
{
// Return stale on fetch failure
var stale = await File.ReadAllTextAsync(cacheFile);
return new MarkdownFetchResult
{
Success = true,
Content = stale
};
}
return fetchResult;
}
finally
{
_fileLock.Release();
}
}
private string GetCacheFilePath(string cacheKey)
=> Path.Combine(_cacheDirectory, $"{cacheKey}.md");
private static string ComputeCacheKey(string url, int blogPostId)
{
var combined = $"{url}_{blogPostId}";
using var sha256 = SHA256.Create();
var bytes = Encoding.UTF8.GetBytes(combined);
var hash = sha256.ComputeHash(bytes);
return Convert.ToHexString(hash);
}
}
In-Memory Storage: Perfect for Demos
SemaphoreSlimKuten näette, tämä toimii seuraavasti:Jos epäonnistut välimuistin sisällön kanssa, palautat tahmean välimuistin (tila-aika-aika-aika-aika!)
public class PostgresMarkdownFetchService : IMarkdownFetchService
{
private readonly MarkdownCacheDbContext _dbContext;
private readonly IHttpClientFactory _httpClientFactory;
private readonly ILogger<PostgresMarkdownFetchService> _logger;
public async Task<MarkdownFetchResult> FetchMarkdownAsync(
string url,
int pollFrequencyHours,
int blogPostId)
{
var cacheKey = GetCacheKey(url, blogPostId);
// Query cache
var cached = await _dbContext.MarkdownCache
.FirstOrDefaultAsync(c => c.CacheKey == cacheKey);
if (cached != null)
{
var age = DateTimeOffset.UtcNow - cached.LastFetchedAt;
if (age.TotalHours < pollFrequencyHours)
{
return new MarkdownFetchResult
{
Success = true,
Content = cached.Content
};
}
}
// Fetch fresh
var fetchResult = await FetchFromUrlAsync(url);
if (fetchResult.Success)
{
if (cached == null)
{
cached = new MarkdownCacheEntry
{
CacheKey = cacheKey,
Url = url,
BlogPostId = blogPostId
};
_dbContext.MarkdownCache.Add(cached);
}
cached.Content = fetchResult.Content;
cached.LastFetchedAt = DateTimeOffset.UtcNow;
await _dbContext.SaveChangesAsync();
}
else if (cached != null)
{
// Return stale
return new MarkdownFetchResult
{
Success = true,
Content = cached.Content
};
}
return fetchResult;
}
}
Virheelle ilman välimuistia, palauttaa virheen
CREATE TABLE markdown_cache (
id SERIAL PRIMARY KEY,
cache_key VARCHAR(128) NOT NULL UNIQUE,
url VARCHAR(2048) NOT NULL,
blog_post_id INTEGER NOT NULL,
content TEXT NOT NULL,
last_fetched_at TIMESTAMP WITH TIME ZONE NOT NULL,
CONSTRAINT ix_markdown_cache_cache_key UNIQUE (cache_key)
);
CREATE INDEX ix_markdown_cache_url_blog_post_id
ON markdown_cache(url, blog_post_id);
Tämä kuvio...
graph TB
subgraph "Load Balancer"
LB[Load Balancer]
end
subgraph "Application Servers"
A1[App Server 1<br/>FetchExtension]
A2[App Server 2<br/>FetchExtension]
A3[App Server 3<br/>FetchExtension]
end
subgraph "Shared Cache"
PG[(PostgreSQL<br/>markdown_cache table)]
end
subgraph "External Content"
R1[Remote URL 1]
R2[Remote URL 2]
R3[Remote URL 3]
end
LB --> A1
LB --> A2
LB --> A3
A1 <-->|Read/Write Cache| PG
A2 <-->|Read/Write Cache| PG
A3 <-->|Read/Write Cache| PG
A1 -.->|Fetch if cache miss| R1
A2 -.->|Fetch if cache miss| R2
A3 -.->|Fetch if cache miss| R3
vanhentunut voimassaoloajan pidentäminen
Vaikka GitHub olisi alhaalla, sivustosi jatkaa työskentelyään välimuistin sisällön kanssa.
dotnet add package mostlylucid.Markdig.FetchExtension
Tiedostopohjainen säilytys: yksinkertainen pysyvyys
# For in-memory (demos/testing)
# Already included in base package
# For file-based storage
# Already included in base package
# For PostgreSQL
dotnet add package mostlylucid.Markdig.FetchExtension.Postgres
# For SQLite
dotnet add package mostlylucid.Markdig.FetchExtension.Sqlite
# For SQL Server
dotnet add package mostlylucid.Markdig.FetchExtension.SqlServer
Tärkeimmät kohdat tässä:Program.cs:
using Mostlylucid.Markdig.FetchExtension;
var builder = WebApplication.CreateBuilder(args);
// Option 1: In-Memory (simplest)
builder.Services.AddInMemoryMarkdownFetch();
// Option 2: File-Based (persists across restarts)
builder.Services.AddFileBasedMarkdownFetch("./markdown-cache");
// Option 3: PostgreSQL (multi-server)
builder.Services.AddPostgresMarkdownFetch(
builder.Configuration.GetConnectionString("MarkdownCache"));
// Option 4: SQLite (single server with DB)
builder.Services.AddSqliteMarkdownFetch("Data Source=markdown-cache.db");
// Option 5: SQL Server (enterprise)
builder.Services.AddSqlServerMarkdownFetch(
builder.Configuration.GetConnectionString("MarkdownCache"));
var app = builder.Build();
// If using database storage, ensure schema exists
if (app.Environment.IsDevelopment())
{
app.Services.EnsureMarkdownCacheDatabase();
}
// Configure the extension with your service provider
FetchMarkdownExtension.ConfigureServiceProvider(app.Services);
app.Run();
langanvarmalle tiedostolle pääsylle
public class MarkdownRenderingService
{
private readonly IServiceProvider _serviceProvider;
private readonly MarkdownFetchPreprocessor _preprocessor;
private readonly MarkdownPipeline _pipeline;
public MarkdownRenderingService(IServiceProvider serviceProvider)
{
_serviceProvider = serviceProvider;
_preprocessor = new MarkdownFetchPreprocessor(serviceProvider);
_pipeline = new MarkdownPipelineBuilder()
.UseAdvancedExtensions()
.UseSyntaxHighlighting()
.UseToc() // Add TOC support for [TOC] markers
.UseYourCustomExtensions()
.Build();
}
public string RenderMarkdown(string markdown)
{
// Step 1: Preprocess to handle fetch tags
var processed = _preprocessor.Preprocess(markdown);
// Step 2: Run through your normal Markdig pipeline
return Markdown.ToHtml(processed, _pipeline);
}
}
Hashes URL + blogikirjoitus ID luoda turvallisia tiedostonimiä
<fetch>Kestää koko sovelluksen uudelleenkäynnistyksetMonipalvelimen käyttöönotossa tämä antaa sinulle välimuistin johdonmukaisuuden kaikissa tapauksissa:**Kaikki palvelimet jakavat saman välimuistin.**Kun Server 1 hakee README:n, palvelimet 2 ja 3 hyötyvät välittömästi välimuistisisällöstä.
Laajennuksen pystyttäminen
Aloittaminen on yksinkertaista.[TOC]Asenna ensin peruspaketti:
# My Document
[TOC]
# Introduction
Content here...
# Getting Started
More content...
## Installation
Details...
Valitse sitten varastonpitäjäsi:
<nav class="ml_toc" aria-label="Table of Contents">
<ul>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#getting-started">Getting Started</a>
<ul>
<li><a href="#installation">Installation</a></li>
</ul>
</li>
</ul>
</nav>
Asetukset ASP.NET-ytimessä
• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •
[TOC cssclass="my-custom-toc"]
Integrointi Markdown Renderointiin
<nav class="my-custom-toc" aria-label="Table of Contents">
<!-- TOC content -->
</nav>
Avain on esikäsittelyvaihe.
**Näin liitän sen blogiini:**Virtaus on:
Markdown sisältäätunnisteet
id="getting-started"id="api-reference"**Kaikki saa custom-laajennuksia, muotoilua jne.**Kehittyneet ominaisuudetul/liSisällysluettelon laatiminen
Paketissa on nyt
erillinen
var pipeline = new MarkdownPipelineBuilder()
.UseAdvancedExtensions()
.UseToc() // Add TOC support - position in pipeline doesn't matter!
.Use<YourOtherExtensions>()
.Build();
**Sisällysluettelon laajennus!**Vaikka se on pakattu noutolaajennuksen rinnalle, se on täysin itsenäinen ja sitä voidaan käyttää yksin.**Asiakirjan otsikoista voi automaattisesti luoda klikattavan sisällystaulukon.**Peruskäyttö:
Näin saadaan pesitty lista kaikista otsikoista, joissa on ankkurilenkit:.UseToc()Omat CSS-luokat:
**Voit määritellä CSS-luokan tyylittelyyn:**Tämä on custom-luokallasi:
**Jos asiakirjasi alkaa H2:lla, TOC pitää H2:ta huipputasona.**ID-sukupolvi[TOC]: Otsikot annetaan automaattisesti ankkureille, jotka yhdistävät:.UseToc()"Aloitetaan" →
Nested Structure
<!-- Failed to fetch content from https://raw.githubusercontent.com/user/repo/main/docs/README.md: HTTP 404: Not Found -->
: Renderaattori rakentaa kunnolla pesivän
./CONTRIBUTING.md → https://github.com/user/repo/blob/main/docs/CONTRIBUTING.md../images/logo.png → https://github.com/user/repo/blob/main/images/logo.pngMahdollinen TOC-tuki:
public class MarkdownLinkRewriter
{
public static string RewriteLinks(string markdown, string sourceUrl)
{
var document = Markdown.Parse(markdown);
var baseUri = GetBaseUri(sourceUrl);
foreach (var link in document.Descendants<LinkInline>())
{
if (IsRelativeLink(link.Url))
{
link.Url = ResolveRelativeLink(baseUri, link.Url);
}
}
using var writer = new StringWriter();
var renderer = new NormalizeRenderer(writer);
renderer.Render(document);
return writer.ToString();
}
private static bool IsRelativeLink(string url)
{
if (string.IsNullOrEmpty(url)) return false;
if (url.StartsWith("http://") || url.StartsWith("https://")) return false;
if (url.StartsWith("#")) return false; // Anchor
if (url.StartsWith("mailto:")) return false;
return true;
}
}
Putken sijainti:
<!-- Failed to fetch content from https://api.example.com/status.md: HTTP request failed: Name or service not known (api.example.com:443) -->
Toisin kuin jotkut Markdigin laajennukset, TOC-laajennus
Ei sillä ole väliä, mihin lisäät sen.putkeen.Laajennus automaattisesti:
Lisää jäsennelmänsä jäsennyslistan alkuun (kohta 0)
<!-- Failed to fetch content from https://example.com/docs.md: HTTP 404: Not Found -->
Renderit ovat sittenkin valmiit, ja ne keräävät otsikoita koko asiakirjasta
Joten voit lisätä
missä tahansa - putken kokoonpanon alussa, keskellä tai lopussa.
{retrieved:format}Tärkeää:{age}TOC-laajennus on täysin riippumaton noutolaajennuksesta.{url}Ne on vain pakattu yhteen mukavuuden vuoksi.{nextrefresh:format}Voit:{pollfrequency}Käytä TOC:tä ilman noutamista{status}Käytä noutamista ilman TOC:täHuomaa:disable="true"TOC-merkki toimii sekä keskeisissä markown-tiedostoissasi että noudetuissa etäsisällöissäsi.
<!-- This will be processed and fetch content -->
<!-- Failed to fetch content from https://example.com/README.md: HTTP 404: Not Found -->
<!-- This will NOT be processed - useful for documentation -->
<!-- Failed to fetch content from https://example.com/README.md: HTTP 404: Not Found -->
Jos haet GitHubilta README:n, joka sisältää
Kun haet etämerkintää (etenkin GitHubista), suhteelliset linkit katkeavat.<fetch>Laajennus voi automaattisesti kirjoittaa ne uusiksi:<fetch-summary>Tämä muuttaa:
<!-- No fetch data available for https://example.com/api/status.md -->
Toteutus käyttää Markdig AST -järjestelmää linkkien kirjoittamiseen:
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
services.AddPostgresMarkdownFetch(connectionString);
var sp = services.BuildServiceProvider();
var eventPublisher = sp.GetRequiredService<IMarkdownFetchEventPublisher>();
// Subscribe to events
eventPublisher.FetchBeginning += (sender, args) =>
{
Console.WriteLine($"Fetching {args.Url}...");
};
eventPublisher.FetchCompleted += (sender, args) =>
{
var source = args.WasCached ? "cache" : "remote";
Console.WriteLine($"Fetched {args.Url} from {source} in {args.Duration.TotalMilliseconds}ms");
};
eventPublisher.FetchFailed += (sender, args) =>
{
Console.WriteLine($"Failed to fetch {args.Url}: {args.ErrorMessage}");
};
}
}
Hae metatietojen yhteenveto
sequenceDiagram
participant MD as Markdown Processor
participant EP as Event Publisher
participant FS as Fetch Service
participant ST as Storage Backend
participant L as Your Listeners
MD->>EP: FetchBeginning
EP->>L: Notify FetchBeginning
EP->>FS: FetchMarkdownAsync(url)
FS->>ST: Check Cache
alt Cache Fresh
ST-->>FS: Cached Content
FS->>EP: FetchCompleted (cached=true)
else Cache Stale/Missing
FS->>FS: HTTP GET
alt Success
FS->>ST: Update Cache
ST-->>FS: OK
FS->>EP: FetchCompleted (cached=false)
else Failure
FS->>EP: FetchFailed
EP->>L: Notify FetchFailed
end
end
EP->>L: Notify FetchCompleted
EP->>L: Notify ContentUpdated
FS-->>MD: MarkdownFetchResult
Note over L: Listeners can be: Logging, Metrics, Telemetry, Webhooks
Tämä tehdään jalankulkijan kanssa:
stateDiagram-v2
[*] --> CheckCache: Fetch Request
CheckCache --> Fresh: Cache exists & age < pollFrequency
CheckCache --> Stale: Cache exists & age >= pollFrequency
CheckCache --> Missing: No cache entry
Fresh --> ReturnCached: Return cached content
ReturnCached --> [*]
Stale --> FetchRemote: Attempt HTTP GET
Missing --> FetchRemote: Attempt HTTP GET
FetchRemote --> UpdateCache: Success
FetchRemote --> HasStale: Failure
UpdateCache --> ReturnFresh: Return new content
ReturnFresh --> [*]
HasStale --> ReturnStale: Return stale cache
HasStale --> ReturnError: No cache available
ReturnStale --> [*]
ReturnError --> [*]
note right of Fresh
pollFrequency = 0
means always stale
end note
note right of HasStale
Stale-while-revalidate
pattern ensures uptime
end note
Aines noudetaan
Dokumentaatiokäsittelyn estäminen
public class MarkdownController : Controller
{
private readonly IMarkdownFetchService _fetchService;
public async Task<IActionResult> InvalidateCache(string url)
{
var removed = await _fetchService.RemoveCachedMarkdownAsync(url);
if (removed)
{
return Ok(new { message = "Cache invalidated" });
}
return NotFound(new { message = "No cache entry found" });
}
}
Kun kirjoitat dokumenttia noutolaajennuksesta (kuten tästä artikkelista!), sinun täytyy näyttää tunnisteet ilman, että niitä käsitellään.
// GitHub webhook notifies of README update
app.MapPost("/webhooks/github", async (
GitHubWebhookPayload payload,
IMarkdownFetchService fetchService) =>
{
if (payload.Repository?.FullName == "user/repo" &&
payload.Commits?.Any(c => c.Modified?.Contains("README.md") == true) == true)
{
var url = "https://raw.githubusercontent.com/user/repo/main/README.md";
await fetchService.RemoveCachedMarkdownAsync(url);
return Results.Ok(new { message = "Cache invalidated" });
}
return Results.Ok(new { message = "No action needed" });
});
Ominaisuus:
public class MarkdownFetchServiceTests
{
[Fact]
public async Task FetchMarkdownAsync_CachesContent()
{
// Arrange
var services = new ServiceCollection();
services.AddLogging();
services.AddInMemoryMarkdownFetch();
var sp = services.BuildServiceProvider();
var fetchService = sp.GetRequiredService<IMarkdownFetchService>();
var url = "https://raw.githubusercontent.com/user/repo/main/README.md";
// Act - First fetch (from network)
var result1 = await fetchService.FetchMarkdownAsync(url, 24, 0);
// Act - Second fetch (from cache)
var result2 = await fetchService.FetchMarkdownAsync(url, 24, 0);
// Assert
Assert.True(result1.Success);
Assert.True(result2.Success);
Assert.Equal(result1.Content, result2.Content);
}
[Fact]
public async Task FetchMarkdownAsync_ReturnsStaleOnFailure()
{
// Arrange
var services = new ServiceCollection();
services.AddLogging();
services.AddInMemoryMarkdownFetch();
var sp = services.BuildServiceProvider();
var fetchService = sp.GetRequiredService<IMarkdownFetchService>();
var url = "https://httpstat.us/200?sleep=100";
// Act - First fetch succeeds
var result1 = await fetchService.FetchMarkdownAsync(url, 0, 0);
// Change URL to fail
var badUrl = "https://httpstat.us/500";
// Act - Second fetch fails, should return stale
var result2 = await fetchService.FetchMarkdownAsync(badUrl, 0, 0);
// Assert
Assert.True(result1.Success);
// Even though fetch failed, we return success with stale content
Assert.True(result2.Success);
}
}
Kirjoitetaan dokumentteja itse laajennuksesta
ConcurrentDictionaryTämä toimii molemmilleIHttpClientFactoryVälimuistikäyttäytyminen noudattaa valtiollisen koneen kaavaa:Palaa heti, verkkoa ei osunut.
Stale Cache
Kadonnut välimuisti
name: Publish Markdig.FetchExtension
on:
push:
tags:
- 'fetchextension-v*.*.*'
permissions:
id-token: write
contents: read
jobs:
build-and-publish:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '9.0.x'
- name: Extract version from tag
id: get_version
run: |
TAG=${GITHUB_REF#refs/tags/fetchextension-v}
echo "VERSION=$TAG" >> $GITHUB_OUTPUT
- name: Build
run: dotnet build Mostlylucid.Markdig.FetchExtension/Mostlylucid.Markdig.FetchExtension.csproj --configuration Release -p:Version=${{ steps.get_version.outputs.VERSION }}
- name: Pack
run: dotnet pack Mostlylucid.Markdig.FetchExtension/Mostlylucid.Markdig.FetchExtension.csproj --configuration Release --no-build -p:PackageVersion=${{ steps.get_version.outputs.VERSION }} --output ./artifacts
- name: Login to NuGet (OIDC)
id: nuget_login
uses: NuGet/login@v1
with:
user: 'mostlylucid'
- name: Publish to NuGet
run: dotnet nuget push ./artifacts/*.nupkg --api-key ${{ steps.nuget_login.outputs.NUGET_API_KEY }} --source https://api.nuget.org/v3/index.json --skip-duplicate
Tätä kaavaa kutsutaan nimellä
Ja se on erinomainen luotettavuuden kannalta.
# My NuGet Package Documentation
Here's the official README from GitHub:
# Umami.Net
## UmamiClient
This is a .NET Core client for the Umami tracking API.
It's based on the Umami Node client, which can be found [here](https://github.com/umami-software/node).
You can see how to set up Umami as a docker
container [here](https://www.mostlylucid.net/blog/usingumamiforlocalanalytics).
You can read more detail about it's creation on my
blog [here](https://www.mostlylucid.net/blog/addingumamitrackingclientfollowup).
To use this client you need the following appsettings.json configuration:
```json
{
"Analytics":{
"UmamiPath" : "https://umamilocal.mostlylucid.net",
"WebsiteId" : "32c2aa31-b1ac-44c0-b8f3-ff1f50403bee"
},
}
Where UmamiPath is the path to your Umami instance and WebsiteId is the id of the website you want to track.
To use the client you need to add the following to your Program.cs:
using Umami.Net;
services.SetupUmamiClient(builder.Configuration);
This will add the Umami client to the services collection.
You can then use the client in two ways:
UmamiClient into your class and call the Track method: // Inject UmamiClient umamiClient
await umamiClient.Track("Search", new UmamiEventData(){{"query", encodedQuery}});
UmamiBackgroundSender to track events in the background (this uses an IHostedService to send events in
the background): // Inject UmamiBackgroundSender umamiBackgroundSender
await umamiBackgroundSender.Track("Search", new UmamiEventData(){{"query", encodedQuery}});
The client will send the event to the Umami API and it will be stored.
The UmamiEventData is a dictionary of key value pairs that will be sent to the Umami API as the event data.
There are additionally more low level methods that can be used to send events to the Umami API.
There's also a convenience method to track a page view. This will send an event to the Umami API with the url set (which counts as a pageview).
await umamiBackgroundSender.TrackPageView("api/search/" + encodedQuery, "searchEvent", eventData: new UmamiEventData(){{"query", encodedQuery}});
await umamiClient.TrackPageView("api/search/" + encodedQuery, "searchEvent", eventData: new UmamiEventData(){{"query", encodedQuery}});
Here we're setting the url to "api/search/" + encodedQuery and the event type to "searchEvent". We're also passing in a dictionary of key value pairs as the event data.
On both the UmamiClient and UmamiBackgroundSender you can call the following method.
Send(UmamiPayload? payload = null, UmamiEventData? eventData = null,
string eventType = "event")
If you don't pass in a UmamiPayload object, the client will create one for you using the WebsiteId from the
appsettings.json.
public UmamiPayload GetPayload(string? url = null, UmamiEventData? data = null)
{
var httpContext = httpContextAccessor.HttpContext;
var request = httpContext?.Request;
var payload = new UmamiPayload
{
Website = settings.WebsiteId,
Data = data,
Url = url ?? httpContext?.Request?.Path.Value,
IpAddress = httpContext?.Connection?.RemoteIpAddress?.ToString(),
UserAgent = request?.Headers["User-Agent"].FirstOrDefault(),
Referrer = request?.Headers["Referer"].FirstOrDefault(),
Hostname = request?.Host.Host,
};
return payload;
}
You can see that this populates the UmamiPayload object with the WebsiteId from the appsettings.json, the Url,
IpAddress, UserAgent, Referrer and Hostname from the HttpContext.
NOTE: eventType can only be "event" or "identify" as per the Umami API.
There's also a service that can be used to pull data from the Umami API. This is a service that allows me to pull data from my Umami instance to use in stuff like sorting posts by popularity etc...
To set it up you need to add a username and password for your umami instance to the Analytics element in your settings file:
"Analytics":{
"UmamiPath" : "https://umami.mostlylucid.net",
"WebsiteId" : "1e3b7657-9487-4857-a9e9-4e1920aa8c42",
"UserName": "admin",
"Password": ""
}
Then in your Program.cs you set up the UmamiDataService as follows:
services.SetupUmamiData(config);
You can then inject the UmamiDataService into your class and use it to pull data from the Umami API.
Now you have the UmamiDataService in your service collection you can start using it!
The methods are all from the Umami API definition you can read about them here: https://umami.is/docs/api/website-stats
All returns are wrapped in an UmamiResults<T> object which has a Success property and a Result property. The
Result property is the object returned from the Umami API.
public record UmamiResult<T>(HttpStatusCode Status, string Message, T? Data);
All requests apart from ActiveUsers have a base request object with two compulsory properties. I added convenience
DateTimes to the base request object to make it easier to set the start and end dates.
public class BaseRequest
{
[QueryStringParameter("startAt", isRequired: true)]
public long StartAt => StartAtDate.ToMilliseconds(); // Timestamp (in ms) of starting date
[QueryStringParameter("endAt", isRequired: true)]
public long EndAt => EndAtDate.ToMilliseconds(); // Timestamp (in ms) of end date
public DateTime StartAtDate { get; set; }
public DateTime EndAtDate { get; set; }
}
The service has the following methods:
This just gets the total number of CURRENT active users on the site
public async Task<UmamiResult<ActiveUsersResponse>> GetActiveUsers()
This returns a bunch of statistics about the site, including the number of users, page views, etc.
public async Task<UmamiResult<StatsResponseModels>> GetStats(StatsRequest statsRequest)
You may set a number of parameters to filter the data returned from the API. For instance using url will return the
stats for a specific URL.
public class StatsRequest : BaseRequest
{
[QueryStringParameter("url")]
public string? Url { get; set; } // Name of URL
[QueryStringParameter("referrer")]
public string? Referrer { get; set; } // Name of referrer
[QueryStringParameter("title")]
public string? Title { get; set; } // Name of page title
[QueryStringParameter("query")]
public string? Query { get; set; } // Name of query
[QueryStringParameter("event")]
public string? Event { get; set; } // Name of event
[QueryStringParameter("host")]
public string? Host { get; set; } // Name of hostname
[QueryStringParameter("os")]
public string? Os { get; set; } // Name of operating system
[QueryStringParameter("browser")]
public string? Browser { get; set; } // Name of browser
[QueryStringParameter("device")]
public string? Device { get; set; } // Name of device (e.g., Mobile)
[QueryStringParameter("country")]
public string? Country { get; set; } // Name of country
[QueryStringParameter("region")]
public string? Region { get; set; } // Name of region/state/province
[QueryStringParameter("city")]
public string? City { get; set; } // Name of city
}
The JSON object Umami returns is as follows.
{
"pageviews": { "value": 5, "change": 5 },
"visitors": { "value": 1, "change": 1 },
"visits": { "value": 3, "change": 2 },
"bounces": { "value": 0, "change": 0 },
"totaltime": { "value": 4, "change": 4 }
}
This is wrapped inside my StatsResponseModel object.
namespace Umami.Net.UmamiData.Models.ResponseObjects;
public class StatsResponseModels
{
public Pageviews pageviews { get; set; }
public Visitors visitors { get; set; }
public Visits visits { get; set; }
public Bounces bounces { get; set; }
public Totaltime totaltime { get; set; }
public class Pageviews
{
public int value { get; set; }
public int prev { get; set; }
}
public class Visitors
{
public int value { get; set; }
public int prev { get; set; }
}
public class Visits
{
public int value { get; set; }
public int prev { get; set; }
}
public class Bounces
{
public int value { get; set; }
public int prev { get; set; }
}
public class Totaltime
{
public int value { get; set; }
public int prev { get; set; }
}
}
Metrics in Umami provide you the number of views for specific types of properties.
One example of these is Events`:
'Events' in Umami are specific items you can track on a site. When tracking events using Umami.Net you can set a number
of properties which are tracked with the event name. For instance here I track Search requests with the URL and the
search term.
await umamiBackgroundSender.Track( "searchEvent", eventData: new UmamiEventData(){{"query", encodedQuery}});
To fetch data about this event you would use the Metrics method:
public async Task<UmamiResult<MetricsResponseModels[]>> GetMetrics(MetricsRequest metricsRequest)
As with the other methods this accepts the MetricsRequest object (with the compulsory BaseRequest properties) and a
number of optional properties to filter the data.
public class MetricsRequest : BaseRequest
{
[QueryStringParameter("type", isRequired: true)]
public MetricType Type { get; set; } // Metrics type
[QueryStringParameter("url")]
public string? Url { get; set; } // Name of URL
[QueryStringParameter("referrer")]
public string? Referrer { get; set; } // Name of referrer
[QueryStringParameter("title")]
public string? Title { get; set; } // Name of page title
[QueryStringParameter("query")]
public string? Query { get; set; } // Name of query
[QueryStringParameter("host")]
public string? Host { get; set; } // Name of hostname
[QueryStringParameter("os")]
public string? Os { get; set; } // Name of operating system
[QueryStringParameter("browser")]
public string? Browser { get; set; } // Name of browser
[QueryStringParameter("device")]
public string? Device { get; set; } // Name of device (e.g., Mobile)
[QueryStringParameter("country")]
public string? Country { get; set; } // Name of country
[QueryStringParameter("region")]
public string? Region { get; set; } // Name of region/state/province
[QueryStringParameter("city")]
public string? City { get; set; } // Name of city
[QueryStringParameter("language")]
public string? Language { get; set; } // Name of language
[QueryStringParameter("event")]
public string? Event { get; set; } // Name of event
[QueryStringParameter("limit")]
public int? Limit { get; set; } = 500; // Number of events returned (default: 500)
}
Here you can see that you can specify a number of properties in the request element to specify what metrics you want to return.
You can also set a Limit property to limit the number of results returned.
For instance to get the event over the past day I mentioned above you would use the following request:
var metricsRequest = new MetricsRequest
{
StartAtDate = DateTime.Now.AddDays(-1),
EndAtDate = DateTime.Now,
Type = MetricType.@event,
Event = "searchEvent"
};
The JSON object returned from the API is as follows:
[
{ "x": "searchEvent", "y": 46 }
]
And again I wrap this in my MetricsResponseModels object.
public class MetricsResponseModels
{
public string x { get; set; }
public int y { get; set; }
}
Where x is the event name and y is the number of times it has been triggered.
One of the most useful metrics is the number of page views. This is the number of times a page has been viewed on the
site. Below is the test I use to get the number of page views over the past 30 days. You'll note the Type parameter is
set as MetricType.url however this is also the default value so you don't need to set it.
[Fact]
public async Task Metrics_StartEnd()
{
var setup = new SetupUmamiData();
var serviceProvider = setup.Setup();
var websiteDataService = serviceProvider.GetRequiredService<UmamiDataService>();
var metrics = await websiteDataService.GetMetrics(new MetricsRequest()
{
StartAtDate = DateTime.Now.AddDays(-30),
EndAtDate = DateTime.Now,
Type = MetricType.url,
Limit = 500
});
Assert.NotNull(metrics);
Assert.Equal( HttpStatusCode.OK, metrics.Status);
}
This returns a MetricsResponse object which has the following JSON structure:
[
{
"x": "/",
"y": 1
},
{
"x": "/blog",
"y": 1
},
{
"x": "/blog/usingumamidataforwebsitestats",
"y": 1
}
]
Where x is the URL and y is the number of times it has been viewed.
This returns the number of page views for a specific URL.
Again here is a test I use for this method:
[Fact]
public async Task PageViews_StartEnd_Day_Url()
{
var setup = new SetupUmamiData();
var serviceProvider = setup.Setup();
var websiteDataService = serviceProvider.GetRequiredService<UmamiDataService>();
var pageViews = await websiteDataService.GetPageViews(new PageViewsRequest()
{
StartAtDate = DateTime.Now.AddDays(-7),
EndAtDate = DateTime.Now,
Unit = Unit.day,
Url = "/blog"
});
Assert.NotNull(pageViews);
Assert.Equal( HttpStatusCode.OK, pageViews.Status);
}
This returns a PageViewsResponse object which has the following JSON structure:
[
{
"date": "2024-09-06 00:00",
"value": 1
}
]
Where date is the date and value is the number of page views, this is repeated for each day in the range specified (
or hour, month, etc. depending on the Unit property).
As with the other methods this accepts the PageViewsRequest object (with the compulsory BaseRequest properties) and
a number of optional properties to filter the data.
public class PageViewsRequest : BaseRequest
{
// Required properties
[QueryStringParameter("unit", isRequired: true)]
public Unit Unit { get; set; } = Unit.day; // Time unit (year | month | hour | day)
[QueryStringParameter("timezone")]
[TimeZoneValidator]
public string Timezone { get; set; }
// Optional properties
[QueryStringParameter("url")]
public string? Url { get; set; } // Name of URL
[QueryStringParameter("referrer")]
public string? Referrer { get; set; } // Name of referrer
[QueryStringParameter("title")]
public string? Title { get; set; } // Name of page title
[QueryStringParameter("host")]
public string? Host { get; set; } // Name of hostname
[QueryStringParameter("os")]
public string? Os { get; set; } // Name of operating system
[QueryStringParameter("browser")]
public string? Browser { get; set; } // Name of browser
[QueryStringParameter("device")]
public string? Device { get; set; } // Name of device (e.g., Mobile)
[QueryStringParameter("country")]
public string? Country { get; set; } // Name of country
[QueryStringParameter("region")]
public string? Region { get; set; } // Name of region/state/province
[QueryStringParameter("city")]
public string? City { get; set; } // Name of city
}
As with the other methods you can set a number of properties to filter the data returned from the API, for instance you
could set the
Country property to get the number of page views from a specific country.
In this site I have some code which lets me use this service to get the number of views each blog page has. In the code
below I take a start and end date and a prefix (which is /blog in my case) and get the number of views for each page
in the blog.
I then cache this data for an hour so I don't have to keep hitting the Umami API.
public class UmamiDataSortService(
UmamiDataService dataService,
IMemoryCache cache)
{
public async Task<List<MetricsResponseModels>?> GetMetrics(DateTime startAt, DateTime endAt, string prefix="" )
{
using var activity = Log.Logger.StartActivity("GetMetricsWithPrefix");
try
{
var cacheKey = $"Metrics_{startAt}_{endAt}_{prefix}";
if (cache.TryGetValue(cacheKey, out List<MetricsResponseModels>? metrics))
{
activity?.AddProperty("CacheHit", true);
return metrics;
}
activity?.AddProperty("CacheHit", false);
var metricsRequest = new MetricsRequest()
{
StartAtDate = startAt,
EndAtDate = endAt,
Type = MetricType.url,
Limit = 500
};
var metricRequest = await dataService.GetMetrics(metricsRequest);
if(metricRequest.Status != HttpStatusCode.OK)
{
return null;
}
var filteredMetrics = metricRequest.Data.Where(x => x.x.StartsWith(prefix)).ToList();
cache.Set(cacheKey, filteredMetrics, TimeSpan.FromHours(1));
activity?.AddProperty("MetricsCount", filteredMetrics?.Count()?? 0);
activity?.Complete();
return filteredMetrics;
}
catch (Exception e)
{
activity?.Complete(LogEventLevel.Error, e);
return null;
}
}
The package is available on NuGet...
Vaikka lähteesi kaatuisi, sivustosi palvelee jatkuvasti välimuistissa olevaa sisältöä.
# Välipalan poisto ja johtaminen
Joskus sinun täytyy mitätöidä välimuisti manuaalisesti:
1. **Tai verkkokoukkujen kautta, kun sisältö muuttuu:**Laajennuksen testaaminen
2. **Laajennus sisältää kattavia testejä.**Rakensin ne näin:
3. **Suorituskykyä koskevia huomioita**Laajennus on suunniteltu suoritukseen:
4. **Rinnakkaissanakirja**- Muistettavat käyttötavat
5. **langan turvalliseen käyttöön**SemaforiSlim
6. **- Tiedostopohjainen käyttää async-lukitusta estääkseen kisaolosuhteet**Tietokantahakemistot
- Kaikilla tietokantatoimittajilla on asianmukaiset hakemistot välimuistin avaimissa
- [HTTP-asiakaspooli](https://www.nuget.org/packages/mostlylucid.Markdig.FetchExtension)
- [- Käytökset](https://www.nuget.org/packages/mostlylucid.Markdig.FetchExtension.Postgres)
- [Tehokkaaseen liitännän uudelleenkäyttöön](https://www.nuget.org/packages/mostlylucid.Markdig.FetchExtension.Sqlite)
- [Asynkki koko matkan](https://www.nuget.org/packages/mostlylucid.Markdig.FetchExtension.SqlServer)
- Ei estäviä puheluita, kaikki on async[Tyypilliset suoritusnumerot kotipalvelimellani:](https://github.com/scottgal/mostlylucidweb/tree/main/Mostlylucid.Markdig.FetchExtension)
Cache-osuma: < 1ms
© 2026 Scott Galloway — Unlicense — All content and source code on this site is free to use, copy, modify, and sell.