Consistent emojis, reworked bookmarks, and navigation ordering

.changeset/gentle-pans-smile.md

@@ -0,0 +1,11 @@
+---
+"@hyperbook/markdown": patch
+"hyperbook": patch
+"hyperbook-studio": patch
+---
+
+Inline the script that loads the light and dark stylesheets. It was a file, and
+nothing can paint before the stylesheets it writes are there, so every first
+paint waited for a round trip to fetch 1.5 kB and then another for the
+stylesheets themselves. It is part of the page now, which measured about 140 ms
+off the first contentful paint of the documentation on a throttled connection.

.changeset/khaki-moles-tap.md

@@ -0,0 +1,20 @@
+---
+"@hyperbook/types": minor
+"@hyperbook/fs": minor
+"@hyperbook/markdown": minor
+"hyperbook": minor
+"hyperbook-studio": minor
+---
+
+Order pages and sections together by `index`. Pages were always rendered before
+the sections of the same level, so a page could not sit after or between them.
+The `index` of a page and the `index` of a section are now one order, and a
+section that ends up between two pages is rendered between them.
+
+The reading order follows the navigation, so the previous and next buttons, the
+breadcrumb and `::pagelist` agree with the sidebar.
+
+A page or a section without an `index` keeps its old place: pages come before
+sections, and a page wins a tie against a section. A book that gives its
+sections an `index` but leaves it off a page will see that page move behind
+those sections. Give the page an `index` to place it.

.changeset/olive-moons-shake.md

@@ -0,0 +1,10 @@
+---
+"@hyperbook/markdown": patch
+"hyperbook": patch
+"hyperbook-studio": patch
+---
+
+Fix the drawers flashing over the page while it loads. A custom element renders
+its children until it is upgraded, and a side drawer is only hidden by its
+shadow root, so the search drawer and the table of contents drawer painted over
+the header and the article until `side-drawer.js` had run.

.changeset/olive-rules-repeat.md

@@ -0,0 +1,43 @@
+---
+"@hyperbook/markdown": minor
+"hyperbook": minor
+"hyperbook-studio": minor
+---
+
+Rework how bookmarks store their label, so bookmarks show what the page shows.
+
+A bookmark label used to be baked into an `onclick` attribute of the bookmark
+button and rendered with `innerHTML`. Labels are now read from the rendered
+heading when a bookmark is saved, and stored as parts instead of markup:
+
+```js
+[{ text: "Getting started " }, { text: "🐧", emoji: "1f427" }];
+```
+
+The bookmark list builds its entries from those parts with the DOM, so nothing
+that was stored is parsed as HTML, and an emoji that is rendered as an image
+stays an image in the bookmark list. Emojis are stored by id, never by URL, so
+bookmarks survive a change of the `basePath`.
+
+This also fixes bookmarking a heading that contains a quote or a backslash.
+Those characters ended up inside a JavaScript string in the `onclick`
+attribute and made the button throw a syntax error.
+
+The bookmark indicator is no longer the 🔖 emoji. Both the button on a heading
+and the marker in the bookmark list are drawn from the stylesheet with a mask,
+so the control looks the same on every platform, takes the color of the
+heading it belongs to in light and dark mode, and does not change when the
+emoji style changes. A bookmarked heading now shows a filled icon instead of only a less
+transparent one.
+
+Breaking:
+
+- `hyperbook.ui.toggleBookmark(key, label)` no longer takes a label:
+  `hyperbook.ui.toggleBookmark(key)`. The label comes from the heading.
+- Bookmark buttons no longer carry an inline `onclick`. They are handled by a
+  delegated listener and carry `data-key`, `data-label`, `aria-label` and
+  `aria-pressed`.
+- Bookmark buttons are empty. The icon comes from `.bookmark` in the
+  stylesheet, so a custom style that replaced the emoji has to be updated.
+- The store moves to version 6. Labels of existing bookmarks are migrated to
+  the new shape, and a plain label is still rendered, so nothing is lost.

.changeset/quiet-jokes-melt.md

@@ -0,0 +1,10 @@
+---
+"@hyperbook/markdown": patch
+"hyperbook": patch
+"hyperbook-studio": patch
+---
+
+Order the pages and the subsections of a section together too. Only the top
+level of the navigation shared one order, so inside a section a subsection was
+still rendered after every page of that section, and the sidebar disagreed with
+the previous and next buttons, which already followed the index.

.changeset/spotty-jars-smash.md

@@ -0,0 +1,11 @@
+---
+"@hyperbook/markdown": patch
+"hyperbook": patch
+"hyperbook-studio": patch
+---
+
+Indent the pages of a section in the navigation. They sat one pixel to the
+right of a page of the level above, so only a hairline told the two levels
+apart. That was harmless while every page came before every section, but a page
+can sit after a section now, and it has to be readable as a page of the level
+above.

.changeset/tricky-donuts-smile.md

@@ -0,0 +1,10 @@
+---
+"@hyperbook/types": patch
+"hyperbook": patch
+"hyperbook-studio": patch
+---
+
+Fix `mailto:` and `tel:` links in markdown. `makeUrl` only passed through URLs
+containing `://` or starting with `data:`, so `[Write us](mailto:a@b.c)` was
+resolved against the base path and became `href="/mailto:a@b.c"`. Any URL with
+a scheme is now passed through untouched.

.changeset/wild-pugs-invite.md

@@ -0,0 +1,13 @@
+---
+"@hyperbook/markdown": patch
+"@hyperbook/types": patch
+"hyperbook": patch
+"hyperbook-studio": patch
+---
+
+Add `elements.emoji.style`. Emojis are drawn by the reader's operating system,
+so the same emoji looks different on Windows, macOS, Android and Linux. Setting
+the style to `twemoji` replaces them with Twemoji images at build time, so a
+hyperbook looks the same on every platform. This covers emojis in the content
+as well as icons from the config, leaves code untouched, and only copies the
+emojis a book actually uses into its output. The default stays `native`.

packages/fs/src/hyperbook.ts

@@ -6,6 +6,7 @@ import {
   HyperbookSection,
   Navigation,
   PageNavigation,
+  sortNavigation,
 } from "@hyperbook/types";
 import { findUp } from "find-up";
 import { vfile } from ".";
@@ -254,17 +255,25 @@ export const getPageList = (
   sections: HyperbookSection[],
   pages: HyperbookPage[],
 ): HyperbookPage[] => {
-  let pageList = [...pages];
+  let pageList: HyperbookPage[] = [];
 
-  for (const section of sections) {
+  // Reading order follows the navigation, so the previous and next buttons
+  // agree with the sidebar when a section sits between two pages.
+  for (const entry of sortNavigation(pages, sections)) {
+    if (entry.type === "page") {
+      pageList.push(entry.page);
+      continue;
+    }
+
+    const section = entry.section;
     // Check if section is virtual or not
     const isVirtual = section.virtual || section.navigation === "virtual";
-    
+
     if (!isVirtual && section.href) {
       // For non-virtual sections with an href, add the index page first
-      const indexPage = section.pages.find(p => p.href === section.href);
-      const otherPages = section.pages.filter(p => p.href !== section.href);
-      
+      const indexPage = section.pages.find((p) => p.href === section.href);
+      const otherPages = section.pages.filter((p) => p.href !== section.href);
+
       if (indexPage) {
         pageList.push(indexPage);
       }

packages/hyperbook/build.ts

@@ -21,6 +21,7 @@ import {
   HyperbookPage,
   HyperbookSection,
   Navigation,
+  isExternalUrl,
 } from "@hyperbook/types";
 import lunr from "lunr";
 import { process as hyperbookProcess } from "@hyperbook/markdown";
@@ -100,6 +101,8 @@ export type PageResultSink = (href: string, result: SinglePageResult) => void;
 export interface SinglePageResult {
   searchDocuments: any[];
   directives: string[];
+  /** Twemoji file names used on this page, without the .svg extension. */
+  emojis: string[];
   /** Absolute paths whose contents were inlined into this page. */
   dependencies: string[];
 }
@@ -134,6 +137,7 @@ export async function buildSingleBookPage(
   const result = await hyperbookProcess(file.markdown.content, ctx);
   const searchDocuments = [...(result.data.searchDocuments || [])];
   const directives = Object.keys(result.data.directives || {});
+  const emojis = [...(result.data.emojis || [])];
 
   for (const generated of (result.data.generatedFiles as any[]) || []) {
     let genDir;
@@ -182,7 +186,12 @@ export async function buildSingleBookPage(
   }
   await fs.writeFile(fileOut, result.value);
 
-  return { searchDocuments, directives, dependencies: [...dependencies] };
+  return {
+    searchDocuments,
+    directives,
+    emojis,
+    dependencies: [...dependencies],
+  };
 }
 
 export async function buildSingleGlossaryPage(
@@ -232,6 +241,7 @@ export async function buildSingleGlossaryPage(
   const result = await hyperbookProcess(file.markdown.content, ctx);
   const searchDocuments = [...(result.data.searchDocuments || [])];
   const directives = Object.keys(result.data.directives || {});
+  const emojis = [...(result.data.emojis || [])];
 
   for (const generated of (result.data.generatedFiles as any[]) || []) {
     let genDir;
@@ -261,7 +271,12 @@ export async function buildSingleGlossaryPage(
   }
   await fs.writeFile(fileOut, result.value);
 
-  return { searchDocuments, directives, dependencies: [...dependencies] };
+  return {
+    searchDocuments,
+    directives,
+    emojis,
+    dependencies: [...dependencies],
+  };
 }
 
 /**
@@ -456,7 +471,7 @@ export function makeBaseCtx(
     version: packageJson.version,
     makeUrl: (p, base, page, options = { versioned: true }) => {
       if (typeof p === "string") {
-        if (p.includes("://") || p.startsWith("data:")) {
+        if (isExternalUrl(p)) {
           return p;
         }
         if (p.endsWith(".md.hbs")) {
@@ -561,6 +576,7 @@ async function runBuild(
   const baseCtx = makeBaseCtx(root, hyperbookJson, basePath, rootProject);
 
   const directives = new Set<string>([]);
+  const emojis = new Set<string>([]);
   const pagesAndSections = await hyperbook.getPagesAndSections(root);
   const pageList = hyperbook.getPageList(
     pagesAndSections.sections,
@@ -589,6 +605,9 @@ async function runBuild(
     for (let directive of pageResult.directives) {
       directives.add(directive);
     }
+    for (let emoji of pageResult.emojis) {
+      emojis.add(emoji);
+    }
     onPage?.(file.path.href || file.path.absolute, pageResult);
 
     if (!process.env.CI) {
@@ -626,6 +645,9 @@ async function runBuild(
     for (let directive of pageResult.directives) {
       directives.add(directive);
     }
+    for (let emoji of pageResult.emojis) {
+      emojis.add(emoji);
+    }
     onPage?.(file.path.href || file.path.absolute, pageResult);
 
     if (!process.env.CI) {
@@ -839,12 +861,44 @@ async function runBuild(
   }
   process.stdout.write("\n");
 
+  if (emojis.size > 0) {
+    const emojiPath = path.join(assetsPath, "emoji");
+    const emojiOut = path.join(assetsOut, "emoji");
+    await mkdir(emojiOut, { recursive: true });
+    i = 1;
+    for (let emoji of emojis) {
+      if (!process.env.CI) {
+        readline.clearLine(process.stdout, 0);
+        readline.cursorTo(process.stdout, 0);
+      }
+      process.stdout.write(
+        `${chalk.blue(`[${prefix}]`)} Copying emojis: [${i++}/${emojis.size}]`,
+      );
+      if (process.env.CI) {
+        process.stdout.write("\n");
+      }
+      try {
+        await cp(
+          path.join(emojiPath, `${emoji}.svg`),
+          path.join(emojiOut, `${emoji}.svg`),
+        );
+      } catch (e) {
+        process.stdout.write("\n");
+        process.stdout.write(
+          `${chalk.red(`[${prefix}]`)} Failed copying emoji: ${emoji}`,
+        );
+        process.stdout.write("\n");
+      }
+    }
+    process.stdout.write("\n");
+  }
+
   const mainAssets = await fs.readdir(assetsPath);
   i = 1;
   for (let asset of mainAssets) {
     const assetPath = path.join(assetsPath, asset);
     const assetOut = path.join(assetsOut, asset);
-    if (!asset.startsWith("directive-")) {
+    if (!asset.startsWith("directive-") && asset !== "emoji") {
       await cp(assetPath, assetOut, {
         recursive: true,
         filter: (src) => {

packages/hyperbook/incremental.ts

@@ -353,6 +353,7 @@ export class IncrementalBuilder {
 
         // Copy any new directive assets
         await this.copyDirectiveAssets(result.directives);
+        await this.copyEmojiAssets(result.emojis);
 
         const changedHref = changedFile.path.href || "/";
 
@@ -423,6 +424,7 @@ export class IncrementalBuilder {
         }
 
         await this.copyDirectiveAssets(result.directives);
+        await this.copyEmojiAssets(result.emojis);
 
         const changedHref = changedFile.path.href || "/glossary";
         await this.refreshSearchIndex();
@@ -487,6 +489,7 @@ export class IncrementalBuilder {
         this.directives.add(directive);
       }
       await this.copyDirectiveAssets(result.directives);
+      await this.copyEmojiAssets(result.emojis);
       return href;
     }
 
@@ -507,6 +510,7 @@ export class IncrementalBuilder {
         this.directives.add(directive);
       }
       await this.copyDirectiveAssets(result.directives);
+      await this.copyEmojiAssets(result.emojis);
       return href;
     }
 
@@ -529,6 +533,27 @@ export class IncrementalBuilder {
     );
   }
 
+  /**
+   * Emojis are copied one file at a time, so a book only ships the Twemoji
+   * images it actually uses.
+   */
+  private async copyEmojiAssets(newEmojis: string[]): Promise<void> {
+    if (newEmojis.length === 0) return;
+    const emojiPath = path.join(__dirname, "assets", "emoji");
+    const emojiOut = path.join(this.assetsOut, "emoji");
+    await fs.mkdir(emojiOut, { recursive: true });
+    for (const emoji of newEmojis) {
+      try {
+        await cp(
+          path.join(emojiPath, `${emoji}.svg`),
+          path.join(emojiOut, `${emoji}.svg`),
+        );
+      } catch {
+        // Emoji has no asset
+      }
+    }
+  }
+
   private async copyDirectiveAssets(newDirectives: string[]): Promise<void> {
     const assetsPath = path.join(__dirname, "assets");
     for (const directive of newDirectives) {