用於將PCRE正則表達式轉換為emacs正則表達式的Elisp機制

Question

如果沒有其他原因，當我輸入'（'我幾乎總是想要一個分組操作符時，我承認喜歡PCRE正則表達式比使用emacs更好的偏見。當然，\\ w和類似的東西比使用它更方便其他等價物。

但是，當然，期望改變emacs的內部結構會很瘋狂。 但我認為應該可以從PCRE表達式轉換為emacs表達式，並執行所有需要的轉換，以便我可以編寫：

(defun my-super-regexp-function ...
   (search-forward (pcre-convert "__\\w: \d+")))

（或類似的）。

有人知道可以做到這一點的elisp庫嗎？

---

編輯：從以下答案中選擇回復...

哇，我喜歡從4天的休假回來，找到一系列有趣的答案！ 我喜歡這兩種解決方案的工作。

最后，看起來exec-a-script和直接elisp版本的解決方案都可以工作，但從純粹的速度和“正確性”方法來看，elisp版本肯定是人們更喜歡的版本（包括我自己） 。

Answer 1

https://github.com/joddie/pcre2el是這個答案的最新版本。

pcre2el或rxt （RegeXp Translator或RegeXp Tools）是一個用於在Emacs中處理正則表達式的實用程序，它基於regexp語法的遞歸下降解析器。 除了將（一部分）PCRE語法轉換為其Emacs等效語言之外，它還可以執行以下操作：

• 將Emacs語法轉換為PCRE
• 將語法轉換為rx ，一種基於S表達式的regexp語法
• 通過以rx形式顯示解析樹並突出顯示相應的代碼塊來解開復雜的正則表達式
• 顯示匹配正則表達式的完整字符串（產品）列表，前提是列表是有限的
• 提供regexp語法的實時字體鎖定（到目前為止僅適用於Elisp緩沖區 - TODO列表中的其他模式）

原始答案的內容如下......

---

這是一個快速而丑陋的Emacs lisp解決方案 （編輯：現在更加永久地定位在這里 ）。 它主要基於pcrepattern手冊頁中的描述，並且通過令牌工作令牌，僅轉換以下構造：

• 括號分組( .. )
• 交替|
• 數字重復{M,N}
• 字符串引用\\Q .. \\E
• 簡單字符轉義： \\a ， \\c ， \\e ， \\f ， \\n ， \\r ， \\t ， \\x和\\ +八進制數字
• 字符類： \\d ， \\D ， \\h ， \\H ， \\s ， \\S ， \\v ， \\V
• \\w和\\W保持不變（使用Emacs自己的單詞和非單詞字符的想法）

它不會對更復雜的PCRE斷言做任何事情，但它會嘗試在字符類中轉換轉義。 在包括類似\\D的字符類的情況下，這通過轉換為具有交替的非捕獲組來完成。

它通過了我為它編寫的測試，但肯定存在錯誤，而且逐個令牌掃描的方法可能很慢。 換句話說，沒有保修。 但也許它會為某些目的做足夠多的工作。 歡迎有興趣的人士改進;-)

(eval-when-compile (require 'cl))

(defvar pcre-horizontal-whitespace-chars
  (mapconcat 'char-to-string
             '(#x0009 #x0020 #x00A0 #x1680 #x180E #x2000 #x2001 #x2002 #x2003
                      #x2004 #x2005 #x2006 #x2007 #x2008 #x2009 #x200A #x202F
                      #x205F #x3000)
             ""))

(defvar pcre-vertical-whitespace-chars
  (mapconcat 'char-to-string
             '(#x000A #x000B #x000C #x000D #x0085 #x2028 #x2029) ""))

(defvar pcre-whitespace-chars
  (mapconcat 'char-to-string '(9 10 12 13 32) ""))

(defvar pcre-horizontal-whitespace
  (concat "[" pcre-horizontal-whitespace-chars "]"))

(defvar pcre-non-horizontal-whitespace
  (concat "[^" pcre-horizontal-whitespace-chars "]"))

(defvar pcre-vertical-whitespace
  (concat "[" pcre-vertical-whitespace-chars "]"))

(defvar pcre-non-vertical-whitespace
  (concat "[^" pcre-vertical-whitespace-chars "]"))

(defvar pcre-whitespace (concat "[" pcre-whitespace-chars "]"))

(defvar pcre-non-whitespace (concat "[^" pcre-whitespace-chars "]"))

(eval-when-compile
  (defmacro pcre-token-case (&rest cases)
    "Consume a token at point and evaluate corresponding forms.

CASES is a list of `cond'-like clauses, (REGEXP FORMS
...). Considering CASES in order, if the text at point matches
REGEXP then moves point over the matched string and returns the
value of FORMS. Returns `nil' if none of the CASES matches."
    (declare (debug (&rest (sexp &rest form))))
    `(cond
      ,@(mapcar
         (lambda (case)
           (let ((token (car case))
                 (action (cdr case)))
             `((looking-at ,token)
               (goto-char (match-end 0))
               ,@action)))
         cases)
      (t nil))))

(defun pcre-to-elisp (pcre)
  "Convert PCRE, a regexp in PCRE notation, into Elisp string form."
  (with-temp-buffer
    (insert pcre)
    (goto-char (point-min))
    (let ((capture-count 0) (accum '())
          (case-fold-search nil))
      (while (not (eobp))
        (let ((translated
               (or
                ;; Handle tokens that are treated the same in
                ;; character classes
                (pcre-re-or-class-token-to-elisp)   

                ;; Other tokens
                (pcre-token-case
                 ("|" "\\|")
                 ("(" (incf capture-count) "\\(")
                 (")" "\\)")
                 ("{" "\\{")
                 ("}" "\\}")

                 ;; Character class
                 ("\\[" (pcre-char-class-to-elisp))

                 ;; Backslash + digits => backreference or octal char?
                 ("\\\\\\([0-9]+\\)"
                  (let* ((digits (match-string 1))
                         (dec (string-to-number digits)))
                    ;; from "man pcrepattern": If the number is
                    ;; less than 10, or if there have been at
                    ;; least that many previous capturing left
                    ;; parentheses in the expression, the entire
                    ;; sequence is taken as a back reference.   
                    (cond ((< dec 10) (concat "\\" digits))
                          ((>= capture-count dec)
                           (error "backreference \\%s can't be used in Emacs regexps"
                                  digits))
                          (t
                           ;; from "man pcrepattern": if the
                           ;; decimal number is greater than 9 and
                           ;; there have not been that many
                           ;; capturing subpatterns, PCRE re-reads
                           ;; up to three octal digits following
                           ;; the backslash, and uses them to
                           ;; generate a data character. Any
                           ;; subsequent digits stand for
                           ;; themselves.
                           (goto-char (match-beginning 1))
                           (re-search-forward "[0-7]\\{0,3\\}")
                           (char-to-string (string-to-number (match-string 0) 8))))))

                 ;; Regexp quoting.
                 ("\\\\Q"
                  (let ((beginning (point)))
                    (search-forward "\\E")
                    (regexp-quote (buffer-substring beginning (match-beginning 0)))))

                 ;; Various character classes
                 ("\\\\d" "[0-9]")
                 ("\\\\D" "[^0-9]")
                 ("\\\\h" pcre-horizontal-whitespace)
                 ("\\\\H" pcre-non-horizontal-whitespace)
                 ("\\\\s" pcre-whitespace)
                 ("\\\\S" pcre-non-whitespace)
                 ("\\\\v" pcre-vertical-whitespace)
                 ("\\\\V" pcre-non-vertical-whitespace)

                 ;; Use Emacs' native notion of word characters
                 ("\\\\[Ww]" (match-string 0))

                 ;; Any other escaped character
                 ("\\\\\\(.\\)" (regexp-quote (match-string 1)))

                 ;; Any normal character
                 ("." (match-string 0))))))
          (push translated accum)))
      (apply 'concat (reverse accum)))))

(defun pcre-re-or-class-token-to-elisp ()
  "Consume the PCRE token at point and return its Elisp equivalent.

Handles only tokens which have the same meaning in character
classes as outside them."
  (pcre-token-case
   ("\\\\a" (char-to-string #x07))  ; bell
   ("\\\\c\\(.\\)"                  ; control character
    (char-to-string
     (- (string-to-char (upcase (match-string 1))) 64)))
   ("\\\\e" (char-to-string #x1b))  ; escape
   ("\\\\f" (char-to-string #x0c))  ; formfeed
   ("\\\\n" (char-to-string #x0a))  ; linefeed
   ("\\\\r" (char-to-string #x0d))  ; carriage return
   ("\\\\t" (char-to-string #x09))  ; tab
   ("\\\\x\\([A-Za-z0-9]\\{2\\}\\)"
    (char-to-string (string-to-number (match-string 1) 16)))
   ("\\\\x{\\([A-Za-z0-9]*\\)}"
    (char-to-string (string-to-number (match-string 1) 16)))))

(defun pcre-char-class-to-elisp ()
  "Consume the remaining PCRE character class at point and return its Elisp equivalent.

Point should be after the opening \"[\" when this is called, and
will be just after the closing \"]\" when it returns."
  (let ((accum '("["))
        (pcre-char-class-alternatives '())
        (negated nil))
    (when (looking-at "\\^")
      (setq negated t)
      (push "^" accum)
      (forward-char))
    (when (looking-at "\\]") (push "]" accum) (forward-char))

    (while (not (looking-at "\\]"))
      (let ((translated
             (or
              (pcre-re-or-class-token-to-elisp)
              (pcre-token-case              
               ;; Backslash + digits => always an octal char
               ("\\\\\\([0-7]\\{1,3\\}\\)"    
                (char-to-string (string-to-number (match-string 1) 8)))

               ;; Various character classes. To implement negative char classes,
               ;; we cons them onto the list `pcre-char-class-alternatives' and
               ;; transform the char class into a shy group with alternation
               ("\\\\d" "0-9")
               ("\\\\D" (push (if negated "[0-9]" "[^0-9]")
                              pcre-char-class-alternatives) "")
               ("\\\\h" pcre-horizontal-whitespace-chars)
               ("\\\\H" (push (if negated
                                  pcre-horizontal-whitespace
                                pcre-non-horizontal-whitespace)
                              pcre-char-class-alternatives) "")
               ("\\\\s" pcre-whitespace-chars)
               ("\\\\S" (push (if negated
                                  pcre-whitespace
                                pcre-non-whitespace)
                              pcre-char-class-alternatives) "")
               ("\\\\v" pcre-vertical-whitespace-chars)
               ("\\\\V" (push (if negated
                                  pcre-vertical-whitespace
                                pcre-non-vertical-whitespace)
                              pcre-char-class-alternatives) "")
               ("\\\\w" (push (if negated "\\W" "\\w") 
                              pcre-char-class-alternatives) "")
               ("\\\\W" (push (if negated "\\w" "\\W") 
                              pcre-char-class-alternatives) "")

               ;; Leave POSIX syntax unchanged
               ("\\[:[a-z]*:\\]" (match-string 0))

               ;; Ignore other escapes
               ("\\\\\\(.\\)" (match-string 0))

               ;; Copy everything else
               ("." (match-string 0))))))
        (push translated accum)))
    (push "]" accum)
    (forward-char)
    (let ((class
           (apply 'concat (reverse accum))))
      (when (or (equal class "[]")
                (equal class "[^]"))
        (setq class ""))
      (if (not pcre-char-class-alternatives)
          class
        (concat "\\(?:"
                class "\\|"
                (mapconcat 'identity
                           pcre-char-class-alternatives
                           "\\|")
                "\\)")))))

Answer 2

我對perlmonks上的perl腳本做了一些小修改（從命令行獲取值）並將其保存為re_pl2el.pl （如下所示）。 然后，以下是將PCRE轉換為elisp正則表達式的不錯的工作，至少對於我測試過的非外來情況。

(defun pcre-to-elre (regex)
  (interactive "MPCRE expression: ")
  (shell-command-to-string (concat "re_pl2el.pl -i -n "
                                   (shell-quote-argument regex))))

(pcre-to-elre "__\\w: \\d+") ;-> "__[[:word:]]: [[:digit:]]+"

它沒有像perl的害羞{N,M}?那樣處理一些“角落”案件{N,M}? 構造，當然不是代碼執行等，但它可能滿足您的需求或是一個良好的起點。 因為你喜歡PCRE我認為你知道足夠的perl來修復你經常使用的任何情況。 如果不讓我知道，我們可以解決它們。

我會更樂意使用一個腳本將正則表達式解析為AST，然后以elisp格式將其吐出（從那以后它也可以用rx格式吐出來），但我找不到任何東西這樣做，看起來像我應該在撰寫論文時做很多工作。 :-)我發現很難相信沒有人這樣做過。

下面是我的“改進版”re_pl2el.pl。 -i表示不對字符串進行雙重轉義， -n表示不打印最終換行符。

#! /usr/bin/perl
#
# File: re_pl2el.pl
# Modified from http://perlmonks.org/?node_id=796020
#
# Description:
#
use strict;
use warnings;

# version 0.4


# TODO
# * wrap converter to function
# * testsuite

#--- flags
my $flag_interactive; # true => no extra escaping of backslashes
if ( int(@ARGV) >= 1 and $ARGV[0] eq '-i' ) {
    $flag_interactive = 1;
    shift @ARGV;
}

if ( int(@ARGV) >= 1 and $ARGV[0] eq '-n' ) {
    shift @ARGV;
} else {
    $\="\n";
}

if ( int(@ARGV) < 1 ) {
    print "usage: $0 [-i] [-n] REGEX";
    exit;
}

my $RE='\w*(a|b|c)\d\(';
$RE='\d{2,3}';
$RE='"(.*?)"';
$RE="\0".'\"\t(.*?)"';
$RE=$ARGV[0];

# print "Perlcode:\t $RE";

#--- encode all \0 chars as escape sequence
$RE=~s#\0#\\0#g;

#--- substitute pairs of backslashes with \0
$RE=~s#\\\\#\0#g;

#--- hide escape sequences of \t,\n,... with
#    corresponding ascii code
my %ascii=(
       t =>"\t",
       n=> "\n"
      );
my $kascii=join "|",keys %ascii;

$RE=~s#\\($kascii)#$ascii{$1}#g;


#---  normalize needless escaping
# e.g.  from /\"/ to /"/, since it's no difference in perl
# but might confuse elisp

$RE=~s#\\"#"#g;

#--- toggle escaping of 'backslash constructs'
my $bsc='(){}|';
$RE=~s#[$bsc]#\\$&#g;  # escape them once
$RE=~s#\\\\##g;        # and erase double-escaping



#--- replace character classes
my %charclass=(
        w => 'word' ,   # TODO: emacs22 already knows \w ???
        d => 'digit',
        s => 'space'
       );

my $kc=join "|",keys %charclass;
$RE=~s#\\($kc)#[[:$charclass{$1}:]]#g;



#--- unhide pairs of backslashes
$RE=~s#\0#\\\\#g;

#--- escaping for elisp string
unless ($flag_interactive){
  $RE=~s#\\#\\\\#g; # ... backslashes
  $RE=~s#"#\\"#g;   # ... quotes
}

#--- unhide escape sequences of \t,\n,...
my %rascii= reverse %ascii;
my $vascii=join "|",keys %rascii;
$RE=~s#($vascii)#\\$rascii{$1}#g;

# print "Elispcode:\t $RE";
print "$RE";
#TODO whats the elisp syntax for \0 ???

Answer 3

以前最接近的工作是對Mx re-builder的擴展，請參閱

http://www.emacswiki.org/emacs/ReBuilder

還是葉文斌關於PDE的工作。

http://cpansearch.perl.org/src/YEWENBIN/Emacs-PDE-0.2.16/lisp/doc/pde.html

Answer 4

可能相關的是visual-regexp-steroids ，它擴展了查詢替換以使用實時預覽，並允許您使用不同的正則表達式后端，包括PCRE。