Implement Pandoc Markdown concise link extension (#20304)

* Implement Pandoc Markdown concise link extension

This implements https://github.com/nim-lang/Nim/issues/20127.
Besides reference to headings we also support doing references
to Nim symbols inside Nim modules.

Markdown:
```
Some heading
------------

Ref. [Some heading].
```

Nim:
```
proc someFunction*() ...

... ## Ref. [someFunction]
```

This is substitution for RST syntax like `` `target`_ ``.
All 3 syntax variants of extension from Pandoc Markdown are supported:
`[target]`, `[target][]`, `[description][target]`.

This PR also fixes clashes in existing files, particularly
conflicts with RST footnote feature, which does not work with
this PR (but there is a plan to adopt a popular [Markdown footnote
extension](https://pandoc.org/MANUAL.html#footnotes) to make footnotes work).

Also the PR fixes a bug that Markdown links did not work when `[...]`
section had a line break.

The implementation is straightforward since link resolution did not
change w.r.t. RST implementation, it's almost only about new syntax
addition. The only essential difference is a possibility to add a custom
link description: form `[description][target]` which does not have an
RST equivalent.

* fix nim 1.0 gotcha

3 files changed, 79 insertions, 2 deletions

diff --git a/nimdoc/testproject/expected/subdir/subdir_b/utils.html b/nimdoc/testproject/expected/subdir/subdir_b/utils.html
index b3d52c4f5..d888a4609 100644
--- a/nimdoc/testproject/expected/subdir/subdir_b/utils.html
+++ b/nimdoc/testproject/expected/subdir/subdir_b/utils.html
@@ -56,6 +56,9 @@
 <ul class="simple"><li><a class="reference" id="next-header-and-so-on_toc" href="#next-header-and-so-on">And so on</a></li>
 </ul></ul><li><a class="reference" id="more-headers_toc" href="#more-headers">More headers</a></li>
 <ul class="simple"><li><a class="reference" id="more-headers-up-to-level-6_toc" href="#more-headers-up-to-level-6">Up to level 6</a></li>
+</ul><li><a class="reference" id="pandoc-markdown_toc" href="#pandoc-markdown">Pandoc Markdown</a></li>
+<ul class="simple"><li><a class="reference" id="pandoc-markdown-link-name-syntax_toc" href="#pandoc-markdown-link-name-syntax">Link name syntax</a></li>
+<li><a class="reference" id="pandoc-markdown-symbols-documentation_toc" href="#pandoc-markdown-symbols-documentation">Symbols documentation</a></li>
 </ul><li>
   <details open>
     <summary><a class="reference reference-toplevel" href="#7" id="57">Types</a></summary>
@@ -229,7 +232,27 @@
                    cmp: proc (x: T; y: K): int {.closure.}): int" href="#binarySearch,openArray[T],K,proc(T,K)">binarySearch proc</a>.</p>
 <p>Ref. type like <a class="reference internal nimdoc" title="type G" href="#G">G</a> and <a class="reference internal nimdoc" title="type G" href="#G">type G</a> and <a class="reference internal nimdoc" title="type G" href="#G">G[T]</a> and <a class="reference internal nimdoc" title="type G" href="#G">type G*[T]</a>.</p>
 <p>Group ref. with capital letters works: <a class="reference internal nimdoc" title="proc fN11 (2 overloads)" href="#fN11-procs-all">fN11</a> or <a class="reference internal nimdoc" title="proc fN11 (2 overloads)" href="#fN11-procs-all">fn11</a> </p>
-Ref. <a class="reference internal nimdoc" title="proc `[]`[T](x: G[T]): T" href="#[],G[T]">[]</a> is the same as <a class="reference internal nimdoc" title="proc `[]`[T](x: G[T]): T" href="#[],G[T]">proc `[]`(G[T])</a> because there are no overloads. The full form: <a class="reference internal nimdoc" title="proc `[]`[T](x: G[T]): T" href="#[],G[T]">proc `[]`*[T](x: G[T]): T</a>Ref. <a class="reference internal nimdoc" title="proc `[]=`[T](a: var G[T]; index: int; value: T)" href="#[]=,G[T],int,T">[]=</a> aka <a class="reference internal nimdoc" title="proc `[]=`[T](a: var G[T]; index: int; value: T)" href="#[]=,G[T],int,T">`[]=`(G[T], int, T)</a>.Ref. <a class="reference internal nimdoc" title="proc $ (2 overloads)" href="#$-procs-all">$</a> aka <a class="reference internal nimdoc" title="proc $ (2 overloads)" href="#$-procs-all">proc $</a> or <a class="reference internal nimdoc" title="proc $ (2 overloads)" href="#$-procs-all">proc `$`</a>.Ref. <a class="reference internal nimdoc" title="proc `$`[T](a: ref SomeType): string" href="#$,ref.SomeType">$(a: ref SomeType)</a>.Ref. <a class="reference internal nimdoc" title="iterator fooBar(a: seq[SomeType]): int" href="#fooBar.i,seq[SomeType]">foo_bar</a> aka <a class="reference internal nimdoc" title="iterator fooBar(a: seq[SomeType]): int" href="#fooBar.i,seq[SomeType]">iterator foo_bar_</a>.Ref. <a class="reference internal nimdoc" title="proc fn[T; U, V: SomeFloat]()" href="#fn">fn[T; U,V: SomeFloat]()</a>.Ref. <a class="reference internal nimdoc" title="proc `'big`(a: string): SomeType" href="#'big,string">'big</a> or <a class="reference internal nimdoc" title="proc `'big`(a: string): SomeType" href="#'big,string">func `'big`</a> or <a class="reference internal nimdoc" title="proc `'big`(a: string): SomeType" href="#'big,string">`'big`(string)</a>.</p>
+Ref. <a class="reference internal nimdoc" title="proc `[]`[T](x: G[T]): T" href="#[],G[T]">[]</a> is the same as <a class="reference internal nimdoc" title="proc `[]`[T](x: G[T]): T" href="#[],G[T]">proc `[]`(G[T])</a> because there are no overloads. The full form: <a class="reference internal nimdoc" title="proc `[]`[T](x: G[T]): T" href="#[],G[T]">proc `[]`*[T](x: G[T]): T</a>Ref. <a class="reference internal nimdoc" title="proc `[]=`[T](a: var G[T]; index: int; value: T)" href="#[]=,G[T],int,T">[]=</a> aka <a class="reference internal nimdoc" title="proc `[]=`[T](a: var G[T]; index: int; value: T)" href="#[]=,G[T],int,T">`[]=`(G[T], int, T)</a>.Ref. <a class="reference internal nimdoc" title="proc $ (2 overloads)" href="#$-procs-all">$</a> aka <a class="reference internal nimdoc" title="proc $ (2 overloads)" href="#$-procs-all">proc $</a> or <a class="reference internal nimdoc" title="proc $ (2 overloads)" href="#$-procs-all">proc `$`</a>.Ref. <a class="reference internal nimdoc" title="proc `$`[T](a: ref SomeType): string" href="#$,ref.SomeType">$(a: ref SomeType)</a>.Ref. <a class="reference internal nimdoc" title="iterator fooBar(a: seq[SomeType]): int" href="#fooBar.i,seq[SomeType]">foo_bar</a> aka <a class="reference internal nimdoc" title="iterator fooBar(a: seq[SomeType]): int" href="#fooBar.i,seq[SomeType]">iterator foo_bar_</a>.Ref. <a class="reference internal nimdoc" title="proc fn[T; U, V: SomeFloat]()" href="#fn">fn[T; U,V: SomeFloat]()</a>.Ref. <a class="reference internal nimdoc" title="proc `'big`(a: string): SomeType" href="#'big,string">'big</a> or <a class="reference internal nimdoc" title="proc `'big`(a: string): SomeType" href="#'big,string">func `'big`</a> or <a class="reference internal nimdoc" title="proc `'big`(a: string): SomeType" href="#'big,string">`'big`(string)</a>.
+<h1><a class="toc-backref" id="pandoc-markdown" href="#pandoc-markdown">Pandoc Markdown</a></h1><p>Now repeat all the auto links of above in Pandoc Markdown Syntax.</p>
+<p>Ref group <a class="reference internal nimdoc" title="proc fn2 (3 overloads)" href="#fn2-procs-all">fn2</a> or specific function like <a class="reference internal nimdoc" title="proc fn2()" href="#fn2">fn2()</a> or <a class="reference internal nimdoc" title="proc fn2(x: int)" href="#fn2,int">fn2(  int  )</a> or <a class="reference internal nimdoc" title="proc fn2(x: int; y: float)" href="#fn2,int,float">fn2(int, float)</a>.</p>
+<p>Ref generics like this: <a class="reference internal nimdoc" title="proc binarySearch[T, K](a: openArray[T]; key: K;
+                   cmp: proc (x: T; y: K): int {.closure.}): int" href="#binarySearch,openArray[T],K,proc(T,K)">binarySearch</a> or <a class="reference internal nimdoc" title="proc binarySearch[T, K](a: openArray[T]; key: K;
+                   cmp: proc (x: T; y: K): int {.closure.}): int" href="#binarySearch,openArray[T],K,proc(T,K)">binarySearch(openArray[T], K, proc (T, K))</a> or <a class="reference internal nimdoc" title="proc binarySearch[T, K](a: openArray[T]; key: K;
+                   cmp: proc (x: T; y: K): int {.closure.}): int" href="#binarySearch,openArray[T],K,proc(T,K)">proc binarySearch(openArray[T], K,  proc (T, K))</a> or in different style: <a class="reference internal nimdoc" title="proc binarySearch[T, K](a: openArray[T]; key: K;
+                   cmp: proc (x: T; y: K): int {.closure.}): int" href="#binarySearch,openArray[T],K,proc(T,K)">proc binarysearch(openarray[T], K, proc(T, K))</a>. Can be combined with export symbols and type parameters: <a class="reference internal nimdoc" title="proc binarySearch[T, K](a: openArray[T]; key: K;
+                   cmp: proc (x: T; y: K): int {.closure.}): int" href="#binarySearch,openArray[T],K,proc(T,K)">binarysearch*[T, K](openArray[T], K, proc (T, K))</a>. With spaces <a class="reference internal nimdoc" title="proc binarySearch[T, K](a: openArray[T]; key: K;
+                   cmp: proc (x: T; y: K): int {.closure.}): int" href="#binarySearch,openArray[T],K,proc(T,K)">binary search</a>.</p>
+<p>Note that <tt class="docutils literal"><span class="pre"><span class="Keyword">proc</span></span></tt> can be used in postfix form: <a class="reference internal nimdoc" title="proc binarySearch[T, K](a: openArray[T]; key: K;
+                   cmp: proc (x: T; y: K): int {.closure.}): int" href="#binarySearch,openArray[T],K,proc(T,K)">binarySearch proc</a>.</p>
+<p>Ref. type like <a class="reference internal nimdoc" title="type G" href="#G">G</a> and <a class="reference internal nimdoc" title="type G" href="#G">type G</a> and <a class="reference internal nimdoc" title="type G" href="#G">G[T]</a> and <a class="reference internal nimdoc" title="type G" href="#G">type G*[T]</a>.</p>
+<p>Group ref. with capital letters works: <a class="reference internal nimdoc" title="proc fN11 (2 overloads)" href="#fN11-procs-all">fN11</a> or <a class="reference internal nimdoc" title="proc fN11 (2 overloads)" href="#fN11-procs-all">fn11</a></p>
+<p>Ref. <a class="reference internal nimdoc" title="proc `[]`[T](x: G[T]): T" href="#[],G[T]">`[]`</a> is the same as <a class="reference internal nimdoc" title="proc `[]`[T](x: G[T]): T" href="#[],G[T]">proc `[]`(G[T])</a> because there are no overloads. The full form: <a class="reference internal nimdoc" title="proc `[]`[T](x: G[T]): T" href="#[],G[T]">proc `[]`*[T](x: G[T]): T</a> Ref. <a class="reference internal nimdoc" title="proc `[]=`[T](a: var G[T]; index: int; value: T)" href="#[]=,G[T],int,T">`[]=`</a> aka <a class="reference internal nimdoc" title="proc `[]=`[T](a: var G[T]; index: int; value: T)" href="#[]=,G[T],int,T">`[]=`(G[T], int, T)</a>. Ref. <a class="reference internal nimdoc" title="proc $ (2 overloads)" href="#$-procs-all">$</a> aka <a class="reference internal nimdoc" title="proc $ (2 overloads)" href="#$-procs-all">proc $</a> or <a class="reference internal nimdoc" title="proc $ (2 overloads)" href="#$-procs-all">proc `$`</a>. Ref. <a class="reference internal nimdoc" title="proc `$`[T](a: ref SomeType): string" href="#$,ref.SomeType">$(a: ref SomeType)</a>. Ref. <a class="reference internal nimdoc" title="iterator fooBar(a: seq[SomeType]): int" href="#fooBar.i,seq[SomeType]">foo_bar</a> aka <a class="reference internal nimdoc" title="iterator fooBar(a: seq[SomeType]): int" href="#fooBar.i,seq[SomeType]">iterator foo_bar_</a>. Ref. <a class="reference internal nimdoc" title="proc fn[T; U, V: SomeFloat]()" href="#fn">fn[T; U,V: SomeFloat]()</a>. Ref. <a class="reference internal nimdoc" title="proc `'big`(a: string): SomeType" href="#'big,string">'big</a> or <a class="reference internal nimdoc" title="proc `'big`(a: string): SomeType" href="#'big,string">func `'big`</a> or <a class="reference internal nimdoc" title="proc `'big`(a: string): SomeType" href="#'big,string">`'big`(string)</a>.</p>
+
+<h2><a class="toc-backref" id="pandoc-markdown-link-name-syntax" href="#pandoc-markdown-link-name-syntax">Link name syntax</a></h2><p>Pandoc Markdown has synax for changing text of links: Ref. <a class="reference internal nimdoc" title="proc `[]`[T](x: G[T]): T" href="#[],G[T]">this proc</a> or <a class="reference internal nimdoc" title="type G" href="#G">another symbol</a>.</p>
+
+<h2><a class="toc-backref" id="pandoc-markdown-symbols-documentation" href="#pandoc-markdown-symbols-documentation">Symbols documentation</a></h2><p>Let us repeat auto links from symbols section below:</p>
+<p>There is also variant <a class="reference internal nimdoc" title="proc f(x: G[string])" href="#f,G[string]">f(G[string])</a>. See also <a class="reference internal nimdoc" title="proc f(x: G[int])" href="#f,G[int]">f(G[int])</a>.</p>
+</p>
     <div class="section" id="7">
   <h1><a class="toc-backref" href="#7">Types</a></h1>
   <dl class="item">
diff --git a/nimdoc/testproject/expected/subdir/subdir_b/utils.idx b/nimdoc/testproject/expected/subdir/subdir_b/utils.idx
index 5c8fee527..007101b37 100644
--- a/nimdoc/testproject/expected/subdir/subdir_b/utils.idx
+++ b/nimdoc/testproject/expected/subdir/subdir_b/utils.idx
@@ -36,3 +36,6 @@ Next header	subdir/subdir_b/utils.html#this-is-now-a-header-next-header	  Next h
 And so on	subdir/subdir_b/utils.html#next-header-and-so-on	   And so on	
 More headers	subdir/subdir_b/utils.html#more-headers	 More headers	
 Up to level 6	subdir/subdir_b/utils.html#more-headers-up-to-level-6	      Up to level 6	
+Pandoc Markdown	subdir/subdir_b/utils.html#pandoc-markdown	 Pandoc Markdown	
+Link name syntax	subdir/subdir_b/utils.html#pandoc-markdown-link-name-syntax	  Link name syntax	
+Symbols documentation	subdir/subdir_b/utils.html#pandoc-markdown-symbols-documentation	  Symbols documentation	
diff --git a/nimdoc/testproject/subdir/subdir_b/utils.nim b/nimdoc/testproject/subdir/subdir_b/utils.nim
index e201a3d38..f535d7f74 100644
--- a/nimdoc/testproject/subdir/subdir_b/utils.nim
+++ b/nimdoc/testproject/subdir/subdir_b/utils.nim
@@ -79,7 +79,9 @@ func fn9*(a: int): int = 42  ## comment
 func fn10*(a: int): int = a  ## comment
 
 # Note capital letter N will be handled correctly in
-# group references like fN11_ or fn11_:
+# group references like fN11_ or fn11_
+# (or [fN11] or [fn11] in Markdown Syntax):
+
 func fN11*() = discard
 func fN11*(x: int) = discard
 
@@ -160,3 +162,52 @@ proc fn*[T; U, V: SomeFloat]() = discard
 ## Ref. `'big`_ or `func \`'big\``_ or `\`'big\`(string)`_.
 
 func `'big`*(a: string): SomeType = discard
+
+##[
+
+Pandoc Markdown
+===============
+
+Now repeat all the auto links of above in Pandoc Markdown Syntax.
+
+Ref group [fn2] or specific function like [fn2()]
+or [fn2(  int  )] or [fn2(int,
+float)].
+
+Ref generics like this: [binarySearch] or [binarySearch(openArray[T], K,
+proc (T, K))] or [proc binarySearch(openArray[T], K,  proc (T, K))] or
+in different style: [proc binarysearch(openarray[T], K, proc(T, K))].
+Can be combined with export symbols and type parameters:
+[binarysearch*[T, K](openArray[T], K, proc (T, K))].
+With spaces [binary search].
+
+Note that `proc` can be used in postfix form: [binarySearch proc].
+
+Ref. type like [G] and [type G] and [G[T]] and [type G*[T]].
+
+Group ref. with capital letters works: [fN11] or [fn11]
+
+Ref. [`[]`] is the same as [proc `[]`(G[T])] because there are no
+overloads. The full form: [proc `[]`*[T](x: G[T]): T]
+Ref. [`[]=`] aka [`[]=`(G[T], int, T)].
+Ref. [$] aka [proc $] or [proc `$`].
+Ref. [$(a: ref SomeType)].
+Ref. [foo_bar] aka [iterator foo_bar_].
+Ref. [fn[T; U,V: SomeFloat]()].
+Ref. ['big] or [func `'big`] or [`'big`(string)].
+
+Link name syntax
+----------------
+
+Pandoc Markdown has synax for changing text of links:
+Ref. [this proc][`[]`] or [another symbol][G[T]].
+
+Symbols documentation
+---------------------
+
+Let us repeat auto links from symbols section below:
+
+There is also variant [f(G[string])].
+See also [f(G[int])].
+
+]##