flex In A Nutshell - Handout 5 - Quick Introduction

Oct 14, 2026

Upload by Muffet

CS143 Summer 2012 handout 05 presents flex, a fast lexical analyzer generator, and explains how to define scanner rules that map input patterns to C code actions. It describes the flex workflow: generating a combined NFA, converting to a minimized DFA, producing lex.yy.c, and using yylex with the lex library or a custom main. The handout outlines flex input structure, Declarations/Definitions/Rules/User subroutines, and key rule pattern operators such as character classes, repetition, anchoring, lookahead, and named definitions.

LanguageEnglish

CategoryPresentations

FormatPDF

Pages5

Reading Time2 min

AccessFree to download

How It Works

Flex Input File Structure

Flex Rules and Actions

Flex Pattern Operators (Character Classes, Repetition, Anchors, Lookahead)

CS143 Summer 2012 Handout 05 June 27, 2012 flex In A Nutshell Handout written by Julie Zelenski with minor edits by Keith Schwarz. flex is a fast lexical analyzer generator. You specify the scanner you want in the form of patterns to match and actions to apply for each token. flex takes your specification and generates a combined NFA to recognize all your patterns, converts it to an equivalent DFA, minimizes the automaton as much as possible, and generates C code that will implement it. flex is similar to another tool, lex, designed by Lesk and Schmidt, and bears many similarities to it. While we will use flex in the course, almost all features we use are also present in the original lex. This handout is designed to give you a quick introduction to the flex tool. This should serve as a useful reference for the first programming assignment. However, you should be aware that in order to complete the first assignment, you may need to use some more advanced features of flex that aren't covered here. To learn more about flex, run info flex, or read the documentation at [http://flex.sourceforge.net/manual/](http://flex.sourceforge.net/manual/) . How It Works flex is designed for use with C code and generates a scanner written in C. The scanner is specified using regular expressions for patterns and C code for the actions. The specification files are traditionally identified by their .l extension. You invoke flex on a .l file and it creates lex.yy .c , a source file containing a wad of unrecognizable C code that implements a FA encoding all your rules and including the code for the actions you specified. The file provides an extern function yylex() that will scan one token. You compile that C file normally, link with the lex library, and you have built a scanner! The scanner reads from stdin and writes to stdout by default. % lex myFile .l % gcc -o scan lex.yy .c -ll % ./scan creates lex.yy .c containing C code for scanner compiles scanner, links with lex lib executes scanner, will read from stdin Linking with the lex library provides a simple main that will repeatedly calls yylex until it reaches EOF. You can also compile and link the scanner into your project and use your own main to control when tokens are scanned. The Makefiles we provide for the projects will execute the compilation steps for you, but it is worthwhile to understand the steps required. 2 A flex Input File flex input files are structured as follows: %{ Declarations %} Definitions %% Rules %% User subroutines The optional Declarations and User subroutines sections are used for ordinary C code that you want copied verbatim to the generated C file. Declarations are copied to the top of the file, user subroutines to the bottom. The optional Definitions section is where you specify options for the scanner and can set up definitions to give names to regular expressions as a simple substitution mechanism that allows for more readable entries in the Rules section that follows. The required Rules section is where you specified the patterns that identify your tokens and the action to perform upon recognizing each token. flex Rules A rule has a regular expression (called the pattern) and an associated set of C statements (called the action) . The idea is that whenever the scanner reads an input sequence that matches a pattern, it executes the action to process it. This is a substantial generalization of regular expressions that allows the tool to be used in many different contexts. In specifying patterns, flex supports a fairly rich set of conveniences (character classes, specific repetition, etc.) beyond our formal language definition of a regular expression. These features don't add expressive power, but simply allow you to construct complicated patterns more succinctly. The table below shows some operators to give you an idea of what is available. For more details, see the web or man pages. Character classes Char

FAQs

What is flex and how does it generate a scanner?

flex generates a scanner by taking a specification of regular-expression patterns and C actions, then building and minimizing automata to recognize the specified tokens. It outputs C code implementing the scanner and provides yylex() to scan tokens.

How is a flex input (.l) file structured?

A flex input file has optional Declarations and User subroutines, an optional Definitions section, and a required Rules section. The Declarations and subroutines are copied verbatim into the generated C file, while Definitions supports named pattern substitution.

How does flex choose among matching patterns when scanning input?

flex finds the longest possible match among all available patterns. If multiple patterns match an equally long sequence, flex uses the pattern that appears first in the file.

Oct 14, 2026

Upload by Muffet