This patch adds a call to PyErr_CheckSignals toSRE_MATCH so that signal handlers can be invoked duringlong regular expression matches. It also adds a newerror return value indicating that an exceptionoccurred in a signal handler during the match, allowingexceptions in the signal handler to propagate up to themain loop.Rationale:Regular expressions can run away inside of the C code.There is no way for Python code to stop the C code fromrunning away, so we attempted to use setrlimit tointerrupt the process when the CPU usage exceeded a limit.When the signal was received, the signal function wastriggered. The sre code does not allow the main loop torun, so the triggered handlers were not called untilthe regular expression finished, if ever. Thisbehaviour makes the interruption by the signal uselessfor the purposes of constraining the running time ofregular expression matches.I am unsure whether the PyErr_CheckSignals islightweight enough to be called inside of the for loopin SRE_MATCH, so I took the conservative approach andonly checked on recursion or match invocation. Ibelieve that the performance hit from this check wouldnot be prohibitive inside of the loop, sincePyErr_CheckSignals does very little work unless thereis a signal to handle.

Logged In: YES user_id=21627Can you give an example for a SRE matching that is so slowthat the user may press Ctrl-C?

Logged In: YES user_id=21627Fredrik, what do you think about this patch?

here is an example (fromhttp://swtch.com/~rsc/regexp/regexp1.html)python -c 'import re; num=25; r=re.compile("a?"*num+"a"*num);r.match("a"*num)'At work I have seen a real world case of a regular expression which ranfor minutes rendering the application unresponsive. We would have beenglad to be able to interrupt the regular expression engine and getting atraceback.

I'm attaching a working patch against 2.5.1 and a short test program.#! /usr/bin/env pythonimport signalimport reimport timedef main():    num=28 # need more than 60s on a 2.4Ghz core 2    r=re.compile("a?"*num+"a"*num)    signal.signal(signal.SIGALRM, signal.default_int_handler)    signal.alarm(1)    stime = time.time()    try:        r.match("a"*num)    except KeyboardInterrupt:        assert time.time()-stime<3    else:        raise RuntimeError("no keyboard interrupt")if __name__=='__main__':    main()

hm. just noticed that calling PyErr_CheckSignals slows down regularexpression matching considerably (50%).I'm adding another patch, which only checks every 4096th iteration forsignals.

Couldn't apply cleanly the patch, as it appears to be a diff in otherformat.Anyway, applied it by hand, and now I attach the correct svn diff.The test cases run ok with this change, and the problem is solved.Regarding the delay introduced, I tested it with:  $ ./python timeit.py -s "import re;r=re.compile('a?a?a?a?a?aaaaa')""r.match('aaaaa')"Trunk:  100000 loops, best of 3: 5.4 usec per loop  100000 loops, best of 3: 5.32 usec per loop  100000 loops, best of 3: 5.41 usec per loopPatch applied:  100000 loops, best of 3: 7.28 usec per loop  100000 loops, best of 3: 6.79 usec per loop  100000 loops, best of 3: 7.00 usec per loopI don't like that. Anyway, I do NOT trust for timing the system whereI'm making the timing, so you may get different results.Suggestions?

./pythonLib/timeit.py -n 1000000 -s "importre;r=re.compile('a?a?a?a?a?aaaaa')" "r.match('aaaaa')" gives me forTrunk:1000000 loops, best of 3: 3.02 usec per loop1000000 loops, best of 3: 2.99 usec per loop1000000 loops, best of 3: 3.01 usec per loopPatched:1000000 loops, best of 3: 3.04 usec per loop1000000 loops, best of 3: 3.04 usec per loop1000000 loops, best of 3: 3.14 usec per loopwhich would be ok, I guess.(This is on a 64bit debian testing with gcc 4.2.3).Can you test with the following:if ((0 == (sigcount & 0xffffffff)) && PyErr_CheckSignals())(i.e. the code will (nearly) not even call PyErr_CheckSignals).I guess this is some c compiler optimization issue (seems like mine doesa better job at optimizing :)

Mind if I assign this to Facundo? Facundo, if you wish to pass this on,just unassign it.

Retried it in a platform where I trust timing, and it proved ok.So, problem solved, no performance impact, all tests pass ok. Commitedinr59862.Thank you all!

I think this is worth backporting to 2.5.2.  This andr60054 are the*only* differences between _sre.c in 2.5.2 and 2.6.

Backported to 2.5.2 asr60576.  (The other deltas are not backported.)