# HG changeset patch # User Michael Pavone # Date 1387838671 28800 # Node ID a1a80af71b054f738b408e84ed0023d18796eae9 # Parent b70be565d54cda1b4315cd84e6f700b64a7bddad Implement onePlus macro. Fix some bugs in the other matching macros. Implement integer literal parsing rules. diff -r b70be565d54c -r a1a80af71b05 modules/parser.tp --- a/modules/parser.tp Mon Dec 23 14:22:20 2013 -0800 +++ b/modules/parser.tp Mon Dec 23 14:44:31 2013 -0800 @@ -66,6 +66,7 @@ matchcall <- quote: (_applyMatch: :tomatch { lm <- left if: (lm matched?) { + orig <- tomatch tomatch <- tomatch from: (lm matchlen) rm <- right if: (rm matched?) { @@ -74,7 +75,7 @@ matched? <- { true } matchlen <- { total } basicYield? <- { true } - yield <- { tomatch from: 0 withLen: total } + yield <- { orig from: 0 withLength: total } } } else: { rm @@ -208,7 +209,7 @@ } } - zeroPlus <- macro: :matchexpr { + _nPlus <- :matchexpr min { funexpr <- false valid <- false mc <- _makeMatchCall: matchexpr @@ -216,6 +217,7 @@ mcall <- mc matchcall quote: :tomatch { cur <- 0 + count <- 0 n <- tomatch byte_length orig <- tomatch match <- true @@ -225,6 +227,7 @@ res <- mcall match <- res matched? if: match { + count <- count + 1 //TODO: Use some kind of lightweight substring wrapper here tomatch <- tomatch from: (res matchlen) if: allBasic? { @@ -242,7 +245,7 @@ cur <- cur + (res matchlen) } } - if: cur > 0 { + if: count >= min { if: allBasic? { #{ matched? <- { true } @@ -266,10 +269,18 @@ } } } else: { - print: "#error Invalid zeroPlus macro call: " . (mc message) . "\n" + print: "#error Invalid nPlus macro call: " . (mc message) . "\n" } } + zeroPlus <- macro: :matchexpr { + _nPlus: matchexpr 0 + } + + onePlus <- macro: :matchexpr { + _nPlus: matchexpr 1 + } + matchOne <- macro: :options { options <- (options value) map: :option { _makeMatchCall: option @@ -404,33 +415,151 @@ _alpha <- charClass: "a-zA-Z" alpha <- zeroPlus: _alpha alphaNum <- zeroPlus: (charClass: "a-zA-Z0-9") + + blockComment <- match: "/*" . (zeroPlus: (matchOne: [(charClass: "^*") "*" . (charClass: "^/")])) . "*/" yield: { false } + hws <- zeroPlus: (matchOne: [ (charClass: " \t") - "/*" . (zeroPlus: (matchOne: [(charClass: "^*") "*" . (charClass: "^/")])) . "*/" + blockComment + ]) + + ws <- zeroPlus: (matchOne: [ + (charClass: " \n\t\r") + "//" . (zeroPlus: (charClass: "^\n")) . "\n" + blockComment ]) + escape <- matchOne: [ + (match: "\\n" yield: {"\n"}) + (match: "\\r" yield: {"\n"}) + (match: "\\t" yield: {"\n"}) + (match: "\\\\" yield: {"\\"}) + (match: "\\\"" yield: {"\""}) + ] + + string <- match: "\"" . Chars . "\"" where: { + Chars <- zeroPlus: (matchOne: [ + (charClass: "^\"\\") + escape + ]) + } yield: { + Chars join: "" + } + + bdigit <- matchOne: [ + (match: "0" yield: {0i64}) + (match: "1" yield: {1i64}) + ] + digit <- matchOne: [ - (match: "0" yield: {0}) - (match: "1" yield: {1}) - (match: "2" yield: {2}) - (match: "3" yield: {3}) - (match: "4" yield: {4}) - (match: "5" yield: {5}) - (match: "6" yield: {6}) - (match: "7" yield: {7}) - (match: "8" yield: {8}) - (match: "9" yield: {9}) + bdigit + (match: "2" yield: {2i64}) + (match: "3" yield: {3i64}) + (match: "4" yield: {4i64}) + (match: "5" yield: {5i64}) + (match: "6" yield: {6i64}) + (match: "7" yield: {7i64}) + (match: "8" yield: {8i64}) + (match: "9" yield: {9i64}) ] - posint <- match: Digits where: { - Digits <- zeroPlus: digit + hdigit <- matchOne: [ + digit + (match: (charClass: "aA") yield: {10i64}) + (match: (charClass: "bB") yield: {11i64}) + (match: (charClass: "cC") yield: {12i64}) + (match: (charClass: "dD") yield: {13i64}) + (match: (charClass: "eE") yield: {14i64}) + (match: (charClass: "fF") yield: {15i64}) + ] + + binary <- match: "0b" . Digits . Suffix where: { + Digits <- onePlus: bdigit + Suffix <- matchOne: [ + (charClass: "ui") . (matchOne: ["8" "16" "32" "64"]) + "" + ] } yield: { num <- Digits fold: 0 with: :acc el { - print: "Element " . el . "\n" - acc * 10 + el + acc * 2i64 + el + } + signed <- true + litbits <- 32 + if: (Suffix length) > 0 { + if: (Suffix from: 0 withLength: 1) = "u" { + signed <- false + } + litbits <- (Suffix from: 1) int32 } #{ litval <- num + signed? <- signed + bits <- litbits + } + } + + decimal <- match: Sign . Digits . Suffix where: { + Sign <- matchOne: ["-" ""] + Digits <- onePlus: digit + Suffix <- matchOne: [ + (charClass: "ui") . (matchOne: ["8" "16" "32" "64"]) + "" + ] + } yield: { + num <- Digits fold: 0 with: :acc el { + acc * 10i64 + el + } + if: Sign = "-" { + num <- 0i64 - num + } + signed <- true + litbits <- 32 + if: (Suffix length) > 0 { + if: (Suffix from: 0 withLength: 1) = "u" { + signed <- false + } + print: (Suffix from: 1) . "\n" + litbits <- (Suffix from: 1) int32 + } + #{ + litval <- num + signed? <- signed + bits <- litbits + } + } + + hex <- match: "0x" . Digits . Suffix where: { + Digits <- onePlus: hdigit + Suffix <- matchOne: [ + (charClass: "ui") . (matchOne: ["8" "16" "32" "64"]) + "" + ] + } yield: { + num <- Digits fold: 0 with: :acc el { + acc * 16i64 + el + } + signed <- true + litbits <- 32 + if: (Suffix length) > 0 { + if: (Suffix from: 0 withLength: 1) = "u" { + signed <- false + } + litbits <- (Suffix from: 1) int32 + } + #{ + litval <- num + signed? <- signed + bits <- litbits + } + } + + testmatchintlit <- :val matchfun { + res <- matchfun: val + if: (res matched?) { + y <- res yield + print: val . " matched with litval " . (y litval) . ", bits " . (y bits) . " and singned? " . (y signed?) . "\n" + } else: { + print: val . " did not match\n" } } @@ -467,11 +596,11 @@ print: "3 did not match\n" } - posintm <- posint: "345" - if: (posintm matched?) { - print: "345 matched with intlit value " . ((posintm yield) litval) . "\n" - } else: { - print: "345 did not match\n" - } + testmatchintlit: "345" :s {decimal: s} + testmatchintlit: "-567" :s {decimal: s} + testmatchintlit: "123u16" :s {decimal: s} + testmatchintlit: "0x20" :s {hex: s} + testmatchintlit: "0x42u64" :s {hex: s} + testmatchintlit: "0b10101" :s {binary: s} } }