9.8 Parsing Monads
9.8.1 A Simple Parsing Monad
Consider the task of parsing. A simple parsing monad is much like a state monad, where the state is the unparsed string. We can represent this exactly as:
newtype Parser a = Parser
{ runParser :: String -> Either String (String, a) }
We again useLeft errto be an error condition. This yields standard instances ofMonadandMonadPlus:
instance Monad Parser where
return a = Parser (\xl -> Right (xl,a)) fail s = Parser (\xl -> Left s) Parser m >>= k = Parser $ \xl -> case m xl of Left s -> Left s Right (xl’, a) -> let Parser n = k a in n xl’
instance MonadPlus Parser where
mzero = Parser (\xl -> Left "mzero")
Parser p ‘mplus‘ Parser q = Parser $ \xl -> case p xl of
Right a -> Right a
Left err -> case q xl of
Right a -> Right a Left _ -> Left err
Now, we want to build up a library of paring “primitives.” The most basic primitive
primitives
is a parser that will read a specific character. This function looks like:
char :: Char -> Parser Char char c = Parser char’
where char’ [] = Left ("expecting " ++ show c ++ " got EOF")
char’ (x:xs)
9.8. PARSING MONADS 145
| otherwise = Left ("expecting " ++ show c ++ " got " ++ show x)
Here, the parser succeeds only if the first character of the input is the expected character.
We can use this parser to build up a parser for the string “Hello”:
helloParser :: Parser String helloParser = do char ’H’ char ’e’ char ’l’ char ’l’ char ’o’ return "Hello"
This shows how easy it is to combine these parsers. We don’t need to worry about the underlying string – the monad takes care of that for us. All we need to do is
combine these parser primatives. We can test this parser by usingrunParserand by runParser
supplying input:
Parsing> runParser helloParser "Hello" Right ("","Hello")
Parsing> runParser helloParser "Hello World!" Right (" World!","Hello")
Parsing> runParser helloParser "hello World!" Left "expecting ’H’ got ’h’"
We can have a slightly more general function, which will match any character fitting a description:
matchChar :: (Char -> Bool) -> Parser Char matchChar c = Parser matchChar’
where matchChar’ [] =
Left ("expecting char, got EOF") matchChar’ (x:xs)
| c x = Right (xs, x) | otherwise =
Left ("expecting char, got " ++ show x)
ciHelloParser = do
c1 <- matchChar (‘elem‘ "Hh") c2 <- matchChar (‘elem‘ "Ee") c3 <- matchChar (‘elem‘ "Ll") c4 <- matchChar (‘elem‘ "Ll") c5 <- matchChar (‘elem‘ "Oo") return [c1,c2,c3,c4,c5]
Of course, we could have used something likematchChar ((==’h’) . toLower), but the above implementation works just as well. We can test this function:
Parsing> runParser ciHelloParser "hELlO world!" Right (" world!","hELlO")
Finally, we can have a function, which will match any character:
anyChar :: Parser Char anyChar = Parser anyChar’
where anyChar’ [] =
Left ("expecting character, got EOF") anyChar’ (x:xs) = Right (xs, x)
On top of these primitives, we usually build some combinators. Themanycombi-
many
nator, for instance, will take a parser that parses entities of typeaand will make it into a parser that parses entities of type[a](this is a Kleene-star operator):
many :: Parser a -> Parser [a] many (Parser p) = Parser many’
where many’ xl =
case p xl of
Left err -> Right (xl, []) Right (xl’,a) ->
let Right (xl’’, rest) = many’ xl’ in Right (xl’’, a:rest)
The idea here is that first we try to apply the given parser,p. If this fails, we succeed but return the empty list. Ifpsucceeds, we recurse and keep trying to applypuntil it fails. We then return the list of successes we’ve accumulated.
In general, there would be many more functions of this sort, and they would be hid- den away in a library, so that users couldn’t actually look inside the Parsertype. However, using them, you could build up, for instance, a parser that parses (non- negative) integers:
9.8. PARSING MONADS 147
int :: Parser Int int = do
t1 <- matchChar isDigit
tr <- many (matchChar isDigit) return (read (t1:tr))
In this function, we first match a digit (theisDigit function comes from the moduleChar/Data.Char) and then match as many more digits as we can. We then
readthe result and return it. We can test this parser as before:
Parsing> runParser int "54" Right ("",54)
*Parsing> runParser int "54abc" Right ("abc",54)
*Parsing> runParser int "a54abc" Left "expecting char, got ’a’"
Now, suppose we want to parse a Haskell-style list ofInts. This becomes somewhat difficult because, at some point, we’re either going to parse a comma or a close brace, but we don’t know when this will happen. This is where the fact thatParseris an instance ofMonadPluscomes in handy: first we try one, then we try the other.
Consider the following code:
intList :: Parser [Int] intList = do
char ’[’
intList’ ‘mplus‘ (char ’]’ >> return []) where intList’ = do
i <- int
r <- (char ’,’ >> intList’) ‘mplus‘ (char ’]’ >> return [])
return (i:r)
The first thing this code does is parse and open brace. Then, usingmplus, it tries mplus
one of two things: parsing usingintList’, or parsing a close brace and returning an empty list.
TheintList’function assumes that we’re not yet at the end of the list, and so it first parses an int. It then parses the rest of the list. However, it doesn’t know whether we’re at the end yet, so it again usesmplus. On the one hand, it tries to parse a comma and then recurse; on the other, it parses a close brace and returns the empty list. Either way, it simply prepends the int it parsed itself to the beginning.
One thing that you should be careful of is the order in which you supply arguments tomplus. Consider the following parser:
tricky =
mplus (string "Hal") (string "Hall")
You might expect this parser to parse both the words “Hal” and “Hall;” however, it only parses the former. You can see this with:
Parsing> runParser tricky "Hal" Right ("","Hal")
Parsing> runParser tricky "Hall" Right ("l","Hal")
This is because it tries to parse “Hal,” which succeeds, and then it doesn’t bother trying to parse “Hall.”
You can attempt to fix this by providing a parser primitive, which detects end-of-file (really, end-of-string) as:
eof :: Parser () eof = Parser eof’
where eof’ [] = Right ([], ())
eof’ xl = Left ("Expecting EOF, got " ++ show (take 10 xl))
You might then rewritetrickyusingeofas:
tricky2 = do
s <- mplus (string "Hal") (string "Hall") eof
return s
But this also doesn’t work, as we can easily see:
Parsing> runParser tricky2 "Hal" Right ("",())
Parsing> runParser tricky2 "Hall" Left "Expecting EOF, got \"l\""
This is because, again, themplusdoesn’t know that it needs to parse the whole input. So, when you provide it with “Hall,” it parses just “Hal” and leaves the last “l” lying around to be parsed later. This causeseofto produce an error message.
The correct way to implement this is:
tricky3 =
9.8. PARSING MONADS 149
eof
return s)
(do s <- string "Hall" eof
return s)
We can see that this works:
Parsing> runParser tricky3 "Hal" Right ("","Hal")
Parsing> runParser tricky3 "Hall" Right ("","Hall")
This works precisely because each side of themplusknows that it must read the end.
In this case, fixing the parser to accept both “Hal” and “Hall” was fairly simple, due to the fact that we assumed we would be reading an end-of-file immediately af- terwards. Unfortunately, if we cannot disambiguate immediately, life becomes signifi- cantly more complicated. This is a general problem in parsing, and has little to do with monadic parsing. The solution most parser libraries (e.g., Parsec, see Section 9.8.2) have adopted is to only recognize “LL(1)” grammars: that means that you must be able to disambiguate the input with a one token look-ahead.
Exercises
Exercise 9.6 Write a parserintListSpace that will parse int lists but will allow arbitrary white space (spaces, tabs or newlines) between the commas and brackets.
Given this monadic parser, it is fairly easy to add information regarding source position. For instance, if we’re parsing a large file, it might be helpful to report the
line number on which an error occurred. We could do this simply by extending the line numbers
Parsertype and by modifying the instances and the primitives:
newtype Parser a = Parser
{ runParser :: Int -> String ->
Either String (Int, String, a) } instance Monad Parser where
return a = Parser (\n xl -> Right (n,xl,a)) fail s = Parser (\n xl -> Left (show n ++
": " ++ s)) Parser m >>= k = Parser $ \n xl -> case m n xl of Left s -> Left s Right (n’, xl’, a) -> let Parser m2 = k a
in m2 n’ xl’
instance MonadPlus Parser where
mzero = Parser (\n xl -> Left "mzero")
Parser p ‘mplus‘ Parser q = Parser $ \n xl -> case p n xl of
Right a -> Right a
Left err -> case q n xl of
Right a -> Right a Left _ -> Left err matchChar :: (Char -> Bool) -> Parser Char matchChar c = Parser matchChar’
where matchChar’ n [] =
Left ("expecting char, got EOF") matchChar’ n (x:xs)
| c x =
Right (n+if x==’\n’ then 1 else 0 , xs, x)
| otherwise =
Left ("expecting char, got " ++ show x)
The definitions forcharandanyCharare not given, since they can be written in terms ofmatchChar. Themanyfunction needs to be modified only to include the new state.
Now, when we run a parser and there is an error, it will tell us which line number contains the error:
Parsing2> runParser helloParser 1 "Hello" Right (1,"","Hello")
Parsing2> runParser int 1 "a54" Left "1: expecting char, got ’a’"
Parsing2> runParser intList 1 "[1,2,3,a]" Left "1: expecting ’]’ got ’1’"
We can use theintListSpaceparser from the prior exercise to see that this does in fact work:
Parsing2> runParser intListSpace 1 "[1 ,2 , 4 \n\n ,a\n]" Left "3: expecting char, got ’a’" Parsing2> runParser intListSpace 1
"[1 ,2 , 4 \n\n\n ,a\n]" Left "4: expecting char, got ’a’"
9.8. PARSING MONADS 151
"[1 ,\n2 , 4 \n\n\n ,a\n]" Left "5: expecting char, got ’a’"
We can see that the line number, on which the error occurs, increases as we add additional newlines before the erroneous “a”.