1.4. Preprocessor
The agreement stated at the end of this section is checked by inspection and by the tests of tests/preproc/, not by a Lean proof.
-
Preproc.Directive[complete] -
Preproc.Line[complete] -
Preproc.Item[complete] -
Preproc.lines[complete] -
Preproc.parse[complete] -
Preproc.group[complete] -
Preproc.condRest[complete]
Preproc is the preprocessor of Core C++, a language of its own that runs before the lexer of Definition 1.1.3. It reads a file as a sequence of lines and never lexes or parses Core C++, so the two languages share no surface. Every Preproc program is also a valid input of the preprocessor of g++.
The terminals of its grammar are whole lines. \textit{Text} stands for any text line, and \textit{NL} ends a directive line.
\begin{array}{lcl} \textit{File} & ::= & \textit{Group} \\ \textit{Group} & ::= & \big(\ \textit{Line}\ \mid\ \textit{Cond}\ \big)^{*} \\ \textit{Line} & ::= & \texttt{\#include}\ \texttt{<}\ \textit{Header}\ \texttt{>}\ \textit{NL}\ \mid\ \texttt{\#define}\ \textit{FlagId}\ \textit{NL}\ \mid\ \textit{Text} \\ \textit{Cond} & ::= & \textit{Test}\ \textit{FlagId}\ \textit{NL}\ \textit{Group}\ \big(\ \texttt{\#else}\ \textit{NL}\ \textit{Group}\ \big)^{?}\ \texttt{\#endif}\ \textit{NL} \\ \textit{Test} & ::= & \texttt{\#ifdef}\ \mid\ \texttt{\#ifndef} \end{array}
Preproc supports conditional compilation on flags, the inclusion of a subset of the standard library and of the STL, and the function assert of <cassert>. A flag has no value and never reaches the code. The design is preproc/ccpp-preproc.md.
Lean code for Definition1.4.1●7 definitions
Associated Lean declarations
-
Preproc.Directive[complete]
-
Preproc.Line[complete]
-
Preproc.Item[complete]
-
Preproc.lines[complete]
-
Preproc.parse[complete]
-
Preproc.group[complete]
-
Preproc.condRest[complete]
-
Preproc.Directive[complete] -
Preproc.Line[complete] -
Preproc.Item[complete] -
Preproc.lines[complete] -
Preproc.parse[complete] -
Preproc.group[complete] -
Preproc.condRest[complete]
-
inductivedefined in Preproc.leancomplete
inductive Preproc.Directive : Type
inductive Preproc.Directive : Type
Constructors
Preproc.Directive.include (h : String) : Preproc.Directive
Preproc.Directive.define (f : String) : Preproc.Directive
Preproc.Directive.ifdef (f : String) : Preproc.Directive
Preproc.Directive.ifndef (f : String) : Preproc.Directive
Preproc.Directive.else_ : Preproc.Directive
Preproc.Directive.endif : Preproc.Directive
-
inductivedefined in Preproc.leancomplete
inductive Preproc.Line : Type
inductive Preproc.Line : Type
Constructors
Preproc.Line.text (s : String) : Preproc.Line
Preproc.Line.dir (d : Preproc.Directive) : Preproc.Line
-
inductivedefined in Preproc.leancomplete
inductive Preproc.Item : Type
inductive Preproc.Item : Type
A parsed group. A conditional keeps whether it has an `#else`, so that the output has one line for each line of the source.
Constructors
Preproc.Item.text (s : String) : Preproc.Item
Preproc.Item.include (loc : Preproc.Loc) (h : String) : Preproc.Item
Preproc.Item.define (f : String) : Preproc.Item
Preproc.Item.cond (neg : Bool) (f : String) (thenI : List Preproc.Item) (hasElse : Bool) (elseI : List Preproc.Item) : Preproc.Item
-
defdefined in Preproc.leancomplete
def Preproc.lines (src : String) : List String
def Preproc.lines (src : String) : List String
The lines of a text, without the empty line after a final newline.
-
defdefined in Preproc.leancomplete
def Preproc.parse (file src : String) : Except String (List Preproc.Item)
def Preproc.parse (file src : String) : Except String (List Preproc.Item)
The items of a file, after the lexical rules on every line.
-
defdefined in Preproc.leancomplete
def Preproc.group : Nat → List (Preproc.Loc × Preproc.Line) → Except String (List Preproc.Item × Preproc.Stop × List (Preproc.Loc × Preproc.Line))
def Preproc.group : Nat → List (Preproc.Loc × Preproc.Line) → Except String (List Preproc.Item × Preproc.Stop × List (Preproc.Loc × Preproc.Line))
A group, up to the end of the input or the next `#else` or `#endif`.
-
defdefined in Preproc.leancomplete
def Preproc.condRest : Nat → Preproc.Loc → Bool → String → List (Preproc.Loc × Preproc.Line) → Except String (List Preproc.Item × Preproc.Stop × List (Preproc.Loc × Preproc.Line))
def Preproc.condRest : Nat → Preproc.Loc → Bool → String → List (Preproc.Loc × Preproc.Line) → Except String (List Preproc.Item × Preproc.Stop × List (Preproc.Loc × Preproc.Line))
The rest of a conditional after its test line, then the rest of the group.
-
Preproc.lexLine[complete] -
Preproc.directive[complete] -
Preproc.checkChars[complete] -
Preproc.isFlag[complete] -
Preproc.words[complete]
Every line satisfies five rules, including the lines of a branch that is not selected and the lines of every header. The C++ preprocessor splices lines and removes comments before it recognises directives (N4659 §5.2, phases 2 to 4), and the rules keep the line structure the same for both.
-
A directive line is exactly one of the forms of the grammar.
-
A text line does not start with
#or with the digraph%:, possibly after blanks, since C++ reads%:as#(N4659 §5.5, Table 1). -
No line contains a backslash,
/*, a single quote or a double quote. -
A flag has the prefix
CCPP_, a nonempty rest of letters, digits and_, and no__, which C++ reserves (N4659 §5.10). -
No word of a text line starts with
CCPP_, so a flag never occurs in the code. A word is a maximal run of letters, digits and_.
Lean code for Definition1.4.2●5 definitions
Associated Lean declarations
-
Preproc.lexLine[complete]
-
Preproc.directive[complete]
-
Preproc.checkChars[complete]
-
Preproc.isFlag[complete]
-
Preproc.words[complete]
-
Preproc.lexLine[complete] -
Preproc.directive[complete] -
Preproc.checkChars[complete] -
Preproc.isFlag[complete] -
Preproc.words[complete]
-
defdefined in Preproc.leancomplete
def Preproc.lexLine (loc : Preproc.Loc) (s : String) : Except String Preproc.Line
def Preproc.lexLine (loc : Preproc.Loc) (s : String) : Except String Preproc.Line
Rules 1 to 5 on one line.
-
defdefined in Preproc.leancomplete
def Preproc.directive (loc : Preproc.Loc) (cs : List Char) : Except String Preproc.Directive
def Preproc.directive (loc : Preproc.Loc) (cs : List Char) : Except String Preproc.Directive
The words of a directive after `#`, and the header of an `#include <h>`.
-
defdefined in Preproc.leancomplete
def Preproc.checkChars (loc : Preproc.Loc) (cs : List Char) : Except String Unit
def Preproc.checkChars (loc : Preproc.Loc) (cs : List Char) : Except String Unit
Rule 3. No backslash, `/*`, `'` or `"`, which could splice lines, hide lines in a comment, or open a literal.
-
defdefined in Preproc.leancomplete
def Preproc.isFlag (f : String) : Bool
def Preproc.isFlag (f : String) : Bool
Rule 4. A flag has the prefix `CCPP_`, a nonempty rest of letters, digits and `_`, and no `__`, since identifiers with `__` are reserved.
-
defdefined in Preproc.leancomplete
def Preproc.words (cs : List Char) : List String
def Preproc.words (cs : List Char) : List String
The words of a line, the maximal runs of letters, digits and `_`.
-
Preproc.run[complete] -
Preproc.runItem[complete] -
Preproc.Item.size[complete] -
Preproc.blank[complete] -
Preproc.translateLines[complete] -
Preproc.translate[complete]
The flag environment \varphi is a finite set of flags, initially the flags given as -D CCPP_X=. The judgement \varphi \vdash G \Rightarrow t, \varphi' gives the output lines t of a group and the environment after it. H(h) is the contents of the Core C++ header h.
\dfrac{}{\varphi \vdash \texttt{\#define}\ F \Rightarrow \varepsilon, \varphi \cup \{F\}}\;\textsf{(P-Define)}
\dfrac{\varphi \vdash H(h) \Rightarrow t, \varphi'}{\varphi \vdash \texttt{\#include}\ \texttt{<}h\texttt{>} \Rightarrow t, \varphi'}\;\textsf{(P-Include)}
\dfrac{F \in \varphi \qquad \varphi \vdash G_1 \Rightarrow t, \varphi'}{\varphi \vdash \texttt{\#ifdef}\ F\ G_1\ \texttt{\#else}\ G_2\ \texttt{\#endif} \Rightarrow t, \varphi'}\;\textsf{(P-IfdefT)}
\dfrac{F \notin \varphi \qquad \varphi \vdash G_2 \Rightarrow t, \varphi'}{\varphi \vdash \texttt{\#ifdef}\ F\ G_1\ \texttt{\#else}\ G_2\ \texttt{\#endif} \Rightarrow t, \varphi'}\;\textsf{(P-IfdefF)}
The directive #ifndef swaps the premises F \in \varphi and F \notin \varphi, and a missing #else stands for an empty G_2. A text line gives itself. An #include gives the output of the header, and Preproc marks those lines as lines of a header. Every other directive line and every line of a branch that is not selected gives an empty line, so a source file without #include keeps its line numbers.
The functions run, runItem and Item.size are partial, so Lean records opaque constants that carry the type and not the body, and no property of them is proved here.
Lean code for Definition1.4.3●6 definitions
Associated Lean declarations
-
Preproc.run[complete]
-
Preproc.runItem[complete]
-
Preproc.Item.size[complete]
-
Preproc.blank[complete]
-
Preproc.translateLines[complete]
-
Preproc.translate[complete]
-
Preproc.run[complete] -
Preproc.runItem[complete] -
Preproc.Item.size[complete] -
Preproc.blank[complete] -
Preproc.translateLines[complete] -
Preproc.translate[complete]
-
opaquedefined in Preproc.leancomplete
opaque Preproc.run (env : Preproc.Env) (φ : List String) : List Preproc.Item → Preproc.M (List Preproc.OutLine × List String)
opaque Preproc.run (env : Preproc.Env) (φ : List String) : List Preproc.Item → Preproc.M (List Preproc.OutLine × List String)
The judgement φ ⊢ G ⇒ t, φ′ on a group, item by item, threading φ.
-
opaquedefined in Preproc.leancomplete
opaque Preproc.runItem (env : Preproc.Env) (φ : List String) : Preproc.Item → Preproc.M (List Preproc.OutLine × List String)
opaque Preproc.runItem (env : Preproc.Env) (φ : List String) : Preproc.Item → Preproc.M (List Preproc.OutLine × List String)
The judgement on one item. ``` ──────────────────────────── (P-Define) φ ⊢ #define F ⇒ ε, φ ∪ {F} φ ⊢ H(h) ⇒ t, φ′ ──────────────────────────── (P-Include) φ ⊢ #include <h> ⇒ t, φ′ F ∈ φ φ ⊢ G₁ ⇒ t, φ′ ─────────────────────────────────────── (P-IfdefT) φ ⊢ #ifdef F G₁ #else G₂ #endif ⇒ t, φ′ F ∉ φ φ ⊢ G₂ ⇒ t, φ′ ─────────────────────────────────────── (P-IfdefF) φ ⊢ #ifdef F G₁ #else G₂ #endif ⇒ t, φ′ ``` The directive `#ifndef` swaps the premises. A text line gives itself, marked when it comes from a header. An `#include` gives the output of the header. Every other directive line and every line of a branch that is not selected gives an empty line. -
opaquedefined in Preproc.leancomplete
opaque Preproc.Item.size : Preproc.Item → Nat
opaque Preproc.Item.size : Preproc.Item → Nat
The number of source lines of a group, for the empty lines of a branch that is not selected.
-
defdefined in Preproc.leancomplete
def Preproc.blank (is : List Preproc.Item) : List Preproc.OutLine
def Preproc.blank (is : List Preproc.Item) : List Preproc.OutLine
-
defdefined in Preproc.leancomplete
def Preproc.translateLines (includeDir : System.FilePath) (φ₀ : List String) (file src : String) : IO (Except String (List Preproc.OutLine))
def Preproc.translateLines (includeDir : System.FilePath) (φ₀ : List String) (file src : String) : IO (Except String (List Preproc.OutLine))
The lines of the output T(p) of a file under the flags φ₀, each marked with whether it comes from a header.
-
defdefined in Preproc.leancomplete
def Preproc.translate (includeDir : System.FilePath) (φ₀ : List String) (file src : String) : IO (Except String String)
def Preproc.translate (includeDir : System.FilePath) (φ₀ : List String) (file src : String) : IO (Except String String)
The output T(p) of a file under the flags φ₀.
The headers of Core C++ are Core C++ code for the Core C++ compiler, in preproc/include, and g++ uses its own headers. Each is a Preproc file whose guard makes it idempotent, as a C++ header is (N4659 §20.5.2.2, paragraph 2).
The headers <vector> and <functional> declare the class templates std::vector and std::function, and <cassert> declares the function assert. Chapter Definition 8.1.1 gives their semantics.
An #include inside a function body puts declarations inside a block, which the grammar of Definition 1.2.1 rejects. Inclusion fails beyond a depth of 64, which bounds a cycle of headers without a guard.
Lean code for Definition1.4.4●1 definition
Associated Lean declarations
-
Preproc.Env[complete]
-
Preproc.Env[complete]
-
structuredefined in Preproc.leancomplete
structure Preproc.Env : Type
structure Preproc.Env : Type
The environment of the translation, the directory of the Core C++ headers and the depth of inclusion, which bounds a cycle of headers without a guard.
Fields
includeDir : System.FilePath
depth : Nat
Take a Preproc program p whose output Core C++ accepts. Compiling p with g++ and running the output in Core C++ give the same exit status under the same -D options.
The preprocessor of C++ selects the same lines as Preproc (N4659 §19.1, paragraphs 11 and 12), and the lexical rules keep flags out of the code. Each Core C++ header agrees with the g++ header of the same name on the names the subset admits, one obligation per header.
The file tests/Preproc.lean checks the argument on the programs of tests/preproc, against g++ -E for the selected lines and against g++ for the exit status. No Lean proof covers it.