Core C++

1.4. Preprocessor🔗

The agreement stated at the end of this section is checked by inspection and by the tests of tests/preproc/, not by a Lean proof.

Definition1.4.1
Group: Syntax, tokens and the parser. (13)
Group member previews
Preview
Definition 1.1.1
Loading preview
Group member preview content is loaded from the rendered-fragment cache.
uses 0
Used by 3
Reverse dependency previews
Preview
Definition 1.4.2
Loading preview
Reverse dependency preview content is loaded from the rendered-fragment cache.
✓L∃∀N

Preproc is the preprocessor of Core C++, a language of its own that runs before the lexer of Definition 1.1.3. It reads a file as a sequence of lines and never lexes or parses Core C++, so the two languages share no surface. Every Preproc program is also a valid input of the preprocessor of g++.

The terminals of its grammar are whole lines. \textit{Text} stands for any text line, and \textit{NL} ends a directive line.

\begin{array}{lcl} \textit{File} & ::= & \textit{Group} \\ \textit{Group} & ::= & \big(\ \textit{Line}\ \mid\ \textit{Cond}\ \big)^{*} \\ \textit{Line} & ::= & \texttt{\#include}\ \texttt{<}\ \textit{Header}\ \texttt{>}\ \textit{NL}\ \mid\ \texttt{\#define}\ \textit{FlagId}\ \textit{NL}\ \mid\ \textit{Text} \\ \textit{Cond} & ::= & \textit{Test}\ \textit{FlagId}\ \textit{NL}\ \textit{Group}\ \big(\ \texttt{\#else}\ \textit{NL}\ \textit{Group}\ \big)^{?}\ \texttt{\#endif}\ \textit{NL} \\ \textit{Test} & ::= & \texttt{\#ifdef}\ \mid\ \texttt{\#ifndef} \end{array}

Preproc supports conditional compilation on flags, the inclusion of a subset of the standard library and of the STL, and the function assert of <cassert>. A flag has no value and never reaches the code. The design is preproc/ccpp-preproc.md.

Lean code for Definition1.4.1●7 definitions
  • inductive(6 constructors)defined in Preproc.lean
    complete
    inductive Preproc.Directive : Type
    inductive Preproc.Directive : Type
    Preproc.Directive.include (h : String) : Preproc.Directive
    Preproc.Directive.define (f : String) : Preproc.Directive
    Preproc.Directive.ifdef (f : String) : Preproc.Directive
    Preproc.Directive.ifndef (f : String) : Preproc.Directive
    Preproc.Directive.else_ : Preproc.Directive
    Preproc.Directive.endif : Preproc.Directive
  • inductive(2 constructors)defined in Preproc.lean
    complete
    inductive Preproc.Line : Type
    inductive Preproc.Line : Type
    Preproc.Line.text (s : String) : Preproc.Line
    Preproc.Line.dir (d : Preproc.Directive) : Preproc.Line
  • inductive(4 constructors)defined in Preproc.lean
    complete
    inductive Preproc.Item : Type
    inductive Preproc.Item : Type
    A parsed group. A conditional keeps whether it has an `#else`, so that the
    output has one line for each line of the source. 
    Preproc.Item.text (s : String) : Preproc.Item
    Preproc.Item.include (loc : Preproc.Loc) (h : String) :
      Preproc.Item
    Preproc.Item.define (f : String) : Preproc.Item
    Preproc.Item.cond (neg : Bool) (f : String)
      (thenI : List Preproc.Item) (hasElse : Bool)
      (elseI : List Preproc.Item) : Preproc.Item
  • defdefined in Preproc.lean
    complete
    def Preproc.lines (src : String) : List String
    def Preproc.lines (src : String) : List String
    The lines of a text, without the empty line after a final newline. 
  • defdefined in Preproc.lean
    complete
    def Preproc.parse (file src : String) : Except String (List Preproc.Item)
    def Preproc.parse (file src : String) :
      Except String (List Preproc.Item)
    The items of a file, after the lexical rules on every line. 
  • defdefined in Preproc.lean
    complete
    def Preproc.group :
      Nat →
        List (Preproc.Loc × Preproc.Line) →
          Except String
            (List Preproc.Item ×
              Preproc.Stop × List (Preproc.Loc × Preproc.Line))
    def Preproc.group :
      Nat →
        List (Preproc.Loc × Preproc.Line) →
          Except String
            (List Preproc.Item ×
              Preproc.Stop ×
                List
                  (Preproc.Loc ×
                    Preproc.Line))
    A group, up to the end of the input or the next `#else` or `#endif`. 
  • defdefined in Preproc.lean
    complete
    def Preproc.condRest :
      Nat →
        Preproc.Loc →
          Bool →
            String →
              List (Preproc.Loc × Preproc.Line) →
                Except String
                  (List Preproc.Item ×
                    Preproc.Stop × List (Preproc.Loc × Preproc.Line))
    def Preproc.condRest :
      Nat →
        Preproc.Loc →
          Bool →
            String →
              List
                  (Preproc.Loc ×
                    Preproc.Line) →
                Except String
                  (List Preproc.Item ×
                    Preproc.Stop ×
                      List
                        (Preproc.Loc ×
                          Preproc.Line))
    The rest of a conditional after its test line, then the rest of the group. 
Definition1.4.2
Group: Syntax, tokens and the parser. (13)
Group member previews
Preview
Definition 1.1.1
Loading preview
Group member preview content is loaded from the rendered-fragment cache.
uses 1used by 1✓L∃∀N

Every line satisfies five rules, including the lines of a branch that is not selected and the lines of every header. The C++ preprocessor splices lines and removes comments before it recognises directives (N4659 §5.2, phases 2 to 4), and the rules keep the line structure the same for both.

  • A directive line is exactly one of the forms of the grammar.

  • A text line does not start with # or with the digraph %:, possibly after blanks, since C++ reads %: as # (N4659 §5.5, Table 1).

  • No line contains a backslash, /*, a single quote or a double quote.

  • A flag has the prefix CCPP_, a nonempty rest of letters, digits and _, and no __, which C++ reserves (N4659 §5.10).

  • No word of a text line starts with CCPP_, so a flag never occurs in the code. A word is a maximal run of letters, digits and _.

Lean code for Definition1.4.2●5 definitions
  • defdefined in Preproc.lean
    complete
    def Preproc.lexLine (loc : Preproc.Loc) (s : String) :
      Except String Preproc.Line
    def Preproc.lexLine (loc : Preproc.Loc)
      (s : String) :
      Except String Preproc.Line
    Rules 1 to 5 on one line. 
  • defdefined in Preproc.lean
    complete
    def Preproc.directive (loc : Preproc.Loc) (cs : List Char) :
      Except String Preproc.Directive
    def Preproc.directive (loc : Preproc.Loc)
      (cs : List Char) :
      Except String Preproc.Directive
    The words of a directive after `#`, and the header of an `#include <h>`. 
  • defdefined in Preproc.lean
    complete
    def Preproc.checkChars (loc : Preproc.Loc) (cs : List Char) :
      Except String Unit
    def Preproc.checkChars (loc : Preproc.Loc)
      (cs : List Char) : Except String Unit
    Rule 3. No backslash, `/*`, `'` or `"`, which could splice lines, hide lines
    in a comment, or open a literal. 
  • defdefined in Preproc.lean
    complete
    def Preproc.isFlag (f : String) : Bool
    def Preproc.isFlag (f : String) : Bool
    Rule 4. A flag has the prefix `CCPP_`, a nonempty rest of letters, digits and
    `_`, and no `__`, since identifiers with `__` are reserved. 
  • defdefined in Preproc.lean
    complete
    def Preproc.words (cs : List Char) : List String
    def Preproc.words (cs : List Char) :
      List String
    The words of a line, the maximal runs of letters, digits and `_`. 
Definition1.4.3
Group: Syntax, tokens and the parser. (13)
Group member previews
Preview
Definition 1.1.1
Loading preview
Group member preview content is loaded from the rendered-fragment cache.
Statement uses 2
Statement dependency previews
Preview
Definition 1.4.1
Loading preview
Statement dependency preview content is loaded from the rendered-fragment cache.
used by 1✓L∃∀N

The flag environment \varphi is a finite set of flags, initially the flags given as -D CCPP_X=. The judgement \varphi \vdash G \Rightarrow t, \varphi' gives the output lines t of a group and the environment after it. H(h) is the contents of the Core C++ header h.

\dfrac{}{\varphi \vdash \texttt{\#define}\ F \Rightarrow \varepsilon, \varphi \cup \{F\}}\;\textsf{(P-Define)}

\dfrac{\varphi \vdash H(h) \Rightarrow t, \varphi'}{\varphi \vdash \texttt{\#include}\ \texttt{<}h\texttt{>} \Rightarrow t, \varphi'}\;\textsf{(P-Include)}

\dfrac{F \in \varphi \qquad \varphi \vdash G_1 \Rightarrow t, \varphi'}{\varphi \vdash \texttt{\#ifdef}\ F\ G_1\ \texttt{\#else}\ G_2\ \texttt{\#endif} \Rightarrow t, \varphi'}\;\textsf{(P-IfdefT)}

\dfrac{F \notin \varphi \qquad \varphi \vdash G_2 \Rightarrow t, \varphi'}{\varphi \vdash \texttt{\#ifdef}\ F\ G_1\ \texttt{\#else}\ G_2\ \texttt{\#endif} \Rightarrow t, \varphi'}\;\textsf{(P-IfdefF)}

The directive #ifndef swaps the premises F \in \varphi and F \notin \varphi, and a missing #else stands for an empty G_2. A text line gives itself. An #include gives the output of the header, and Preproc marks those lines as lines of a header. Every other directive line and every line of a branch that is not selected gives an empty line, so a source file without #include keeps its line numbers.

The functions run, runItem and Item.size are partial, so Lean records opaque constants that carry the type and not the body, and no property of them is proved here.

Lean code for Definition1.4.3●6 definitions
  • opaquedefined in Preproc.lean
    complete
    opaque Preproc.run (env : Preproc.Env) (φ : List String) :
      List Preproc.Item → Preproc.M (List Preproc.OutLine × List String)
    opaque Preproc.run (env : Preproc.Env)
      (φ : List String) :
      List Preproc.Item →
        Preproc.M
          (List Preproc.OutLine × List String)
    The judgement φ ⊢ G ⇒ t, φ′ on a group, item by item, threading φ. 
  • opaquedefined in Preproc.lean
    complete
    opaque Preproc.runItem (env : Preproc.Env) (φ : List String) :
      Preproc.Item → Preproc.M (List Preproc.OutLine × List String)
    opaque Preproc.runItem (env : Preproc.Env)
      (φ : List String) :
      Preproc.Item →
        Preproc.M
          (List Preproc.OutLine × List String)
    The judgement on one item.
    
    ```
    ──────────────────────────── (P-Define)
    φ ⊢ #define F ⇒ ε, φ ∪ {F}
    
    φ ⊢ H(h) ⇒ t, φ′
    ──────────────────────────── (P-Include)
    φ ⊢ #include <h> ⇒ t, φ′
    
    F ∈ φ    φ ⊢ G₁ ⇒ t, φ′
    ─────────────────────────────────────── (P-IfdefT)
    φ ⊢ #ifdef F G₁ #else G₂ #endif ⇒ t, φ′
    
    F ∉ φ    φ ⊢ G₂ ⇒ t, φ′
    ─────────────────────────────────────── (P-IfdefF)
    φ ⊢ #ifdef F G₁ #else G₂ #endif ⇒ t, φ′
    ```
    
    The directive `#ifndef` swaps the premises. A text line gives itself, marked
    when it comes from a header. An `#include` gives the output of the header.
    Every other directive line and every line of a branch that is not selected
    gives an empty line. 
  • opaquedefined in Preproc.lean
    complete
    opaque Preproc.Item.size : Preproc.Item → Nat
    opaque Preproc.Item.size : Preproc.Item → Nat
    The number of source lines of a group, for the empty lines of a branch that
    is not selected. 
  • defdefined in Preproc.lean
    complete
    def Preproc.blank (is : List Preproc.Item) : List Preproc.OutLine
    def Preproc.blank (is : List Preproc.Item) :
      List Preproc.OutLine
  • defdefined in Preproc.lean
    complete
    def Preproc.translateLines (includeDir : System.FilePath) (φ₀ : List String)
      (file src : String) : IO (Except String (List Preproc.OutLine))
    def Preproc.translateLines
      (includeDir : System.FilePath)
      (φ₀ : List String) (file src : String) :
      IO
        (Except String (List Preproc.OutLine))
    The lines of the output T(p) of a file under the flags φ₀, each marked with
    whether it comes from a header. 
  • defdefined in Preproc.lean
    complete
    def Preproc.translate (includeDir : System.FilePath) (φ₀ : List String)
      (file src : String) : IO (Except String String)
    def Preproc.translate
      (includeDir : System.FilePath)
      (φ₀ : List String) (file src : String) :
      IO (Except String String)
    The output T(p) of a file under the flags φ₀. 
Definition1.4.4
Group: Syntax, tokens and the parser. (13)
Group member previews
Preview
Definition 1.1.1
Loading preview
Group member preview content is loaded from the rendered-fragment cache.
uses 1
Used by 3
Reverse dependency previews
Preview
Definition 1.4.3
Loading preview
Reverse dependency preview content is loaded from the rendered-fragment cache.
✓L∃∀N

The headers of Core C++ are Core C++ code for the Core C++ compiler, in preproc/include, and g++ uses its own headers. Each is a Preproc file whose guard makes it idempotent, as a C++ header is (N4659 §20.5.2.2, paragraph 2).

The headers <vector> and <functional> declare the class templates std::vector and std::function, and <cassert> declares the function assert. Chapter Definition 8.1.1 gives their semantics.

An #include inside a function body puts declarations inside a block, which the grammar of Definition 1.2.1 rejects. Inclusion fails beyond a depth of 64, which bounds a cycle of headers without a guard.

Lean code for Definition1.4.4●1 definition
  • structure(2 fields)defined in Preproc.lean
    complete
    structure Preproc.Env : Type
    structure Preproc.Env : Type
    The environment of the translation, the directory of the Core C++ headers
    and the depth of inclusion, which bounds a cycle of headers without a guard. 
    includeDir : System.FilePath
    depth : Nat
Theorem1.4.5
Group: Syntax, tokens and the parser. (13)
Group member previews
Preview
Definition 1.1.1
Loading preview
Group member preview content is loaded from the rendered-fragment cache.
Statement uses 3
Statement dependency previews
Preview
Definition 1.4.2
Loading preview
Statement dependency preview content is loaded from the rendered-fragment cache.
used by 0XL∃∀N

Take a Preproc program p whose output Core C++ accepts. Compiling p with g++ and running the output in Core C++ give the same exit status under the same -D options.

Proof for Theorem 1.4.5
uses 0

The preprocessor of C++ selects the same lines as Preproc (N4659 §19.1, paragraphs 11 and 12), and the lexical rules keep flags out of the code. Each Core C++ header agrees with the g++ header of the same name on the names the subset admits, one obligation per header.

The file tests/Preproc.lean checks the argument on the programs of tests/preproc, against g++ -E for the selected lines and against g++ for the exit status. No Lean proof covers it.