+++++++++++++++++++++ CPython results, 2017 +++++++++++++++++++++ This page lists benchmarks which became faster in CPython. Optimizations ============= 2016-12-14: speedup method calls -------------------------------- .. XXX results of the commit must validated on speed-python. Optimization: `Speedup method calls 1.2x `_, `commit f2392133 `_. +---------------------+---------------------------+------------------------------+ | Benchmark | 2016-12-01 (27580c1fb5e8) | 2017-01-01 (67e1aa0b58be) | +=====================+===========================+==============================+ | call_method | 14.1 ms | 11.2 ms: 1.26x faster (-21%) | +---------------------+---------------------------+------------------------------+ | call_method_slots | 13.9 ms | 11.1 ms: 1.25x faster (-20%) | +---------------------+---------------------------+------------------------------+ | call_method_unknown | 16.0 ms | 14.3 ms: 1.12x faster (-11%) | +---------------------+---------------------------+------------------------------+ 2016-04-22: pymalloc allocator ------------------------------ Optimization: `PyMem_Malloc() now uses the fast pymalloc allocator `_, `commit f5c4b990 `_. Changes of at least 5%: +----------------------+---------------------------+------------------------------+ | Benchmark | 2016-04-21 (5439fc4901db) | 2016-04-22 (f5c4b99034fa) | +======================+===========================+==============================+ | unpickle_list | 10.4 us | 7.64 us: 1.36x faster (-27%) | +----------------------+---------------------------+------------------------------+ | json_dumps | 28.0 ms | 25.2 ms: 1.11x faster (-10%) | +----------------------+---------------------------+------------------------------+ | unpickle_pure_python | 741 us | 678 us: 1.09x faster (-9%) | +----------------------+---------------------------+------------------------------+ | unpickle | 33.9 us | 31.3 us: 1.08x faster (-8%) | +----------------------+---------------------------+------------------------------+ | meteor_contest | 197 ms | 183 ms: 1.08x faster (-7%) | +----------------------+---------------------------+------------------------------+ | mako | 36.9 ms | 34.3 ms: 1.07x faster (-7%) | +----------------------+---------------------------+------------------------------+ | pathlib | 41.0 ms | 38.4 ms: 1.07x faster (-6%) | +----------------------+---------------------------+------------------------------+ | call_method_slots | 14.8 ms | 13.9 ms: 1.07x faster (-6%) | +----------------------+---------------------------+------------------------------+ | telco | 19.5 ms | 18.3 ms: 1.07x faster (-6%) | +----------------------+---------------------------+------------------------------+ | scimark_lu | 413 ms | 388 ms: 1.07x faster (-6%) | +----------------------+---------------------------+------------------------------+ | nqueens | 221 ms | 207 ms: 1.07x faster (-6%) | +----------------------+---------------------------+------------------------------+ | fannkuch | 937 ms | 882 ms: 1.06x faster (-6%) | +----------------------+---------------------------+------------------------------+ | regex_compile | 319 ms | 301 ms: 1.06x faster (-6%) | +----------------------+---------------------------+------------------------------+ | raytrace | 1.16 sec | 1.09 sec: 1.06x faster (-5%) | +----------------------+---------------------------+------------------------------+ | pickle_pure_python | 1.11 ms | 1.05 ms: 1.05x faster (-5%) | +----------------------+---------------------------+------------------------------+ | genshi_text | 70.1 ms | 66.6 ms: 1.05x faster (-5%) | +----------------------+---------------------------+------------------------------+ 2015-12-07: Optimize ElementTree.iterparse(), xml_etree_iterparse ----------------------------------------------------------------- .. XXX results of the commit must validated on speed-python. Optimization: `Issue #25638: Optimized ElementTree.iterparse(); it is now 2x faster `_, `commit 9ec5e25f2 `_. +---------------------+---------------------------+-----------------------------+ | Benchmark | 2015-12-01 (df144092a340) | 2016-01-01 (71db90356390) | +=====================+===========================+=============================+ | xml_etree_iterparse | 423 ms | 206 ms: 2.05x faster (-51%) | +---------------------+---------------------------+-----------------------------+ 2015-09-19: PGO uses test suite, pidigits ----------------------------------------- Optimization: `Issue #24915: Add Clang support to PGO builds and use the test suite for profile data `_, `commit 7188a3ef `_. Changes of at least 5%: +----------------------+--------------------------------------+--------------------------------------+ | Benchmark | 2015-09-18 (4b363e270108) | 2015-09-18_22-13 (7188a3efe07b) | +======================+======================================+======================================+ | pickle | 33.7 us | 26.4 us: 1.28x faster (-22%) | +----------------------+--------------------------------------+--------------------------------------+ | pidigits | 332 ms | 286 ms: 1.16x faster (-14%) | +----------------------+--------------------------------------+--------------------------------------+ | pickle_list | 9.90 us | 8.84 us: 1.12x faster (-11%) | +----------------------+--------------------------------------+--------------------------------------+ | unpickle | 37.2 us | 33.3 us: 1.12x faster (-11%) | +----------------------+--------------------------------------+--------------------------------------+ | unpickle_list | 11.1 us | 9.95 us: 1.11x faster (-10%) | +----------------------+--------------------------------------+--------------------------------------+ | regex_dna | 330 ms | 297 ms: 1.11x faster (-10%) | +----------------------+--------------------------------------+--------------------------------------+ | regex_effbot | 6.43 ms | 5.80 ms: 1.11x faster (-10%) | +----------------------+--------------------------------------+--------------------------------------+ | pickle_dict | 69.3 us | 64.1 us: 1.08x faster (-8%) | +----------------------+--------------------------------------+--------------------------------------+ | mako | 39.1 ms | 36.2 ms: 1.08x faster (-7%) | +----------------------+--------------------------------------+--------------------------------------+ | call_simple | 12.2 ms | 11.6 ms: 1.05x faster (-5%) | +----------------------+--------------------------------------+--------------------------------------+ | genshi_xml | 175 ms | 166 ms: 1.05x faster (-5%) | +----------------------+--------------------------------------+--------------------------------------+ Changes of at least 5%, sadly two benchmarks also became slower: +----------------------+--------------------------------------+--------------------------------------+ | Benchmark | 2015-09-18_14-32-master-4b363e270108 | 2015-09-18_22-13-master-7188a3efe07b | +======================+======================================+======================================+ | unpickle_pure_python | 776 us | 821 us: 1.06x slower (+6%) | +----------------------+--------------------------------------+--------------------------------------+ | regex_v8 | 49.5 ms | 52.6 ms: 1.06x slower (+6%) | +----------------------+--------------------------------------+--------------------------------------+ 2015-05-30: C implementation of collections.OrderedDict, html5lib ----------------------------------------------------------------- .. XXX results of the commit must validated on speed-python. Optimization: `Issue #16991: Add a C implementation of collections.OrderedDict `_, `commit 96c6af9b `_. +-----------------+---------------------------+-----------------------------+ | Benchmark | 2015-05-02 (3b4d30a27bd6) | 2015-06-01 (41874c570cf3) | +=================+===========================+=============================+ | html5lib | 285 ms | 233 ms: 1.23x faster (-19%) | +-----------------+---------------------------+-----------------------------+ 2015-05-23: C implementation of functools.lru_cache(), sympy ------------------------------------------------------------ Optimization: `Issue #14373: Added C implementation of functools.lru_cache() `_, `commit 1c858c35 `_. Changes of at least 5%: +-----------------+--------------------------------------+--------------------------------------+ | Benchmark | 2015-05-23_19-15-master-c70908558d8e | 2015-05-23_19-42-master-1c858c352b8c | +=================+======================================+======================================+ | sympy_expand | 1.45 sec | 1.14 sec: 1.27x faster (-21%) | +-----------------+--------------------------------------+--------------------------------------+ | sympy_sum | 308 ms | 247 ms: 1.25x faster (-20%) | +-----------------+--------------------------------------+--------------------------------------+ | sympy_str | 621 ms | 500 ms: 1.24x faster (-19%) | +-----------------+--------------------------------------+--------------------------------------+ | sympy_integrate | 54.2 ms | 45.7 ms: 1.19x faster (-16%) | +-----------------+--------------------------------------+--------------------------------------+ | scimark_lu | 497 ms | 471 ms: 1.06x faster (-5%) | +-----------------+--------------------------------------+--------------------------------------+ ``pickle_dict`` is seen as 1.06x slower, but since pickle doesn't use functools.lru_cache(), the change is ignored in the table. Slowdown ======== 2016-09-11: regex_compile ------------------------- Slowdown: `convert re flags to (much friendlier) IntFlag constants (issue #28082) `_, `commit f93395bc `_. +---------------+---------------------------+----------------------------+-----------------------------+ | Benchmark | 2016-04-01 (6b6abd4cf10e) | 2016-07-01 (355048970b2a) | 2016-10-01 (78a111c7d867) | +===============+===========================+============================+=============================+ | regex_compile | 339 ms | 309 ms: 1.10x faster (-9%) | 383 ms: 1.13x slower (+13%) | +---------------+---------------------------+----------------------------+-----------------------------+ Timeline ======== April, 2016 -> May, 2016 ------------------------ 2016-04-01 .. 2016-05-01: +----------------+---------------------------+-------------------------------+ | Benchmark | 2016-04-01 (dcfebb32e277) | 2016-05-01 (f1e2671fdf88) | +================+===========================+===============================+ | nqueens | 255 ms | 207 ms: 1.23x faster (-19%) | +----------------+---------------------------+-------------------------------+ | raytrace | 1.31 sec | 1.09 sec: 1.19x faster (-16%) | +----------------+---------------------------+-------------------------------+ | float | 290 ms | 243 ms: 1.19x faster (-16%) | +----------------+---------------------------+-------------------------------+ | chaos | 273 ms | 235 ms: 1.16x faster (-14%) | +----------------+---------------------------+-------------------------------+ | hexiom | 21.0 ms | 18.6 ms: 1.13x faster (-11%) | +----------------+---------------------------+-------------------------------+ | deltablue | 16.4 ms | 14.6 ms: 1.12x faster (-11%) | +----------------+---------------------------+-------------------------------+ | go | 557 ms | 502 ms: 1.11x faster (-10%) | +----------------+---------------------------+-------------------------------+ | nbody | 254 ms | 232 ms: 1.10x faster (-9%) | +----------------+---------------------------+-------------------------------+ call_method ----------- Timeline 2016-04-01 .. 2017-01-01: +---------------------+---------------------------+------------------------------+------------------------------+--------------------------------------+ | Benchmark | 2016-04-01 (6b6abd4cf10e) | 2016-07-01 (355048970b2a) | 2016-10-01 (78a111c7d867) | 2017-01-01 (67e1aa0b58be) | +=====================+===========================+==============================+==============================+======================================+ | call_method | 15.8 ms | 14.9 ms: 1.06x faster (-6%) | 14.1 ms: 1.13x faster (-11%) | 11.2 ms: 1.42x faster (-29%) | +---------------------+---------------------------+------------------------------+------------------------------+--------------------------------------+ | call_method_slots | 15.7 ms | 15.2 ms: 1.03x faster (-3%) | 14.0 ms: 1.13x faster (-11%) | 11.1 ms: 1.42x faster (-29%) | +---------------------+---------------------------+------------------------------+------------------------------+--------------------------------------+ | call_method_unknown | 17.7 ms | 15.9 ms: 1.11x faster (-10%) | 15.6 ms: 1.13x faster (-11%) | 14.3 ms: 1.23x faster (-19%) | +---------------------+---------------------------+------------------------------+------------------------------+--------------------------------------+ crypto_pyaes ------------ .. XXX failed to get any significant diff on smithers between dcfebb32e277a68b9c6582e6a0484e6d (2016-04-01) and f1e2671fdf88fce8a367ee63aba4a (2016-05-01) (diff: -1%) .. XXX related to PyMem_Malloc()? .. XXX failed to reproduce on speed-python (no LTO, no PGO) old_commit = dcfebb32e277a68b9c6582e6a0484e6d new_commit = f1e2671fdf88fce8a367ee63aba4a +--------------+---------------------+----------------------------+ | Benchmark | 2016-04-01 (master) | 2016-05-01 (master) | +==============+=====================+============================+ | crypto_pyaes | 226 ms | 205 ms: 1.10x faster (-9%) | +--------------+---------------------+----------------------------+ 2016-03-01 .. 2016-06-01: +--------------+---------------------------+-----------------------------+ | Benchmark | 2016-03-01 (13d09afff127) | 2016-06-01 (d80ab7d94578) | +==============+===========================+=============================+ | crypto_pyaes | 231 ms | 199 ms: 1.16x faster (-14%) | +--------------+---------------------------+-----------------------------+ json_loads ---------- Progress on 21 months, 2015-01-01 .. 2016-10-01: +------------+---------------------------+------------------------------+ | Benchmark | 2015-01-01 (52074ac866eb) | 2016-10-01 (78a111c7d867) | +============+===========================+==============================+ | json_loads | 64.0 us | 56.6 us: 1.13x faster (-11%) | +------------+---------------------------+------------------------------+ logging_silent -------------- .. (1) 2016-01 .. 2016-02 -- not reproduced on speed-python (2) 2016-04 .. 2016-05 (3) 2017-01 .. 2017-02 -- not reproduced on speed-python +----------------+---------------------------+-----------------------------+ | Benchmark | 2016-01-01 (899b72cee21c) | 2016-07-01 (355048970b2a) | +================+===========================+=============================+ | logging_silent | 718 ns | 606 ns: 1.18x faster (-16%) | +----------------+---------------------------+-----------------------------+ pickle ------ pickle, 2016-08-02 .. 2016-09-08: .. XXX unable to reproduce on speed-python (no LTO, no PGO) 133138a284be1985ebd9ec9014f1306b9a425 .. 10427f44852b6e872034061421a8890902b8f diff: -6% .. commit 711120d8fd0445b33655101d72b0f576646bff9f faster import? http://bugs.python.org/issue22557 +-----------+---------------------------+------------------------------+ | Benchmark | 2016-08-02 (133138a284be) | 2016-09-08 (10427f44852b) | +===========+===========================+==============================+ | pickle | 25.5 us | 21.4 us: 1.19x faster (-16%) | +-----------+---------------------------+------------------------------+ pickle dict/list: .. XXX pickle_dict diff is very small (-5 us), hard to bisect +-------------+---------------------------+------------------------------+ | Benchmark | 2016-04-01 (6b6abd4cf10e) | 2016-10-01 (78a111c7d867) | +=============+===========================+==============================+ | pickle_dict | 64.5 us | 57.7 us: 1.12x faster (-11%) | +-------------+---------------------------+------------------------------+ | pickle_list | 9.06 us | 7.79 us: 1.16x faster (-14%) | +-------------+---------------------------+------------------------------+ unpickle: +-----------+---------------------------+------------------------------+ | Benchmark | 2015-07-01 (d7982beca93c) | 2015-10-01 (30b7138fe12b) | +===========+===========================+==============================+ | unpickle | 36.9 us | 32.8 us: 1.13x faster (-11%) | +-----------+---------------------------+------------------------------+ python_startup -------------- 2015-04-01 .. 2015-10-01: +------------------------+---------------------------+-----------------------------+ | Benchmark | 2015-04-01 (4fd929b43121) | 2015-10-01 (30b7138fe12b) | +========================+===========================+=============================+ | python_startup | 16.4 ms | 17.2 ms: 1.05x slower (+5%) | +------------------------+---------------------------+-----------------------------+ | python_startup_no_site | 8.65 ms | 8.90 ms: 1.03x slower (+3%) | +------------------------+---------------------------+-----------------------------+ 2016-04-01 .. 2017-01-01: +------------------------+---------------------------+------------------------------+ | Benchmark | 2016-04-01 (6b6abd4cf10e) | 2017-01-01 (67e1aa0b58be) | +========================+===========================+==============================+ | python_startup | 17.3 ms | 14.5 ms: 1.20x faster (-16%) | +------------------------+---------------------------+------------------------------+ | python_startup_no_site | 8.89 ms | 8.39 ms: 1.06x faster (-6%) | +------------------------+---------------------------+------------------------------+ regex_compile ------------- +---------------+---------------------------+----------------------------+-----------------------------+ | Benchmark | 2016-04-01 (6b6abd4cf10e) | 2016-07-01 (355048970b2a) | 2016-10-01 (78a111c7d867) | +===============+===========================+============================+=============================+ | regex_compile | 339 ms | 309 ms: 1.10x faster (-9%) | 383 ms: 1.13x slower (+13%) | +---------------+---------------------------+----------------------------+-----------------------------+ telco ----- +-----------+---------------------------+-----------------------------+-----------------------------+------------------------------+------------------------------+ | Benchmark | 2016-01-01 (899b72cee21c) | 2016-04-01 (6b6abd4cf10e) | 2016-07-01 (355048970b2a) | 2016-10-01 (78a111c7d867) | 2017-03-31 (cdcac039fb44) | +===========+===========================+=============================+=============================+==============================+==============================+ | telco | 19.6 ms | 19.2 ms: 1.02x faster (-2%) | 18.3 ms: 1.08x faster (-7%) | 15.1 ms: 1.30x faster (-23%) | 13.9 ms: 1.41x faster (-29%) | +-----------+---------------------------+-----------------------------+-----------------------------+------------------------------+------------------------------+ scimark ------- .. XXX unable to reproduce on smithers (without LTO nor PGO), ef158c3ced3f (2017-02-01) .. 1cf2a809b1eb4f9fc8ac3ccc97424586892d1e1a (2017-02-28) 2016-10-01 .. 2017-03-31: +-------------+---------------------------+-----------------------------+-----------------------------+ | Benchmark | 2016-10-01 (78a111c7d867) | 2017-01-01 (67e1aa0b58be) | 2017-03-31 (cdcac039fb44) | +=============+===========================+=============================+=============================+ | scimark_lu | 423 ms | 378 ms: 1.12x faster (-11%) | 318 ms: 1.33x faster (-25%) | +-------------+---------------------------+-----------------------------+-----------------------------+ | scimark_sor | 426 ms | 403 ms: 1.06x faster (-5%) | 375 ms: 1.14x faster (-12%) | +-------------+---------------------------+-----------------------------+-----------------------------+ sqlalchemy_declarative ---------------------- +------------------------+---------------------------+-----------------------------+ | Benchmark | 2014-10-01 (5a789f7eaf81) | 2015-10-01 (30b7138fe12b) | +========================+===========================+=============================+ | sqlalchemy_declarative | 345 ms | 301 ms: 1.15x faster (-13%) | +------------------------+---------------------------+-----------------------------+ sympy ----- 2016-04-01 .. 2016-10-01: +-----------------+---------------------------+------------------------------+------------------------------+ | Benchmark | 2016-04-01 (6b6abd4cf10e) | 2016-07-01 (355048970b2a) | 2016-10-01 (78a111c7d867) | +=================+===========================+==============================+==============================+ | sympy_expand | 1.10 sec | 1.01 sec: 1.09x faster (-8%) | 942 ms: 1.17x faster (-14%) | +-----------------+---------------------------+------------------------------+------------------------------+ | sympy_integrate | 46.6 ms | 42.9 ms: 1.09x faster (-8%) | 41.2 ms: 1.13x faster (-11%) | +-----------------+---------------------------+------------------------------+------------------------------+ | sympy_sum | 247 ms | 233 ms: 1.06x faster (-6%) | 199 ms: 1.24x faster (-19%) | +-----------------+---------------------------+------------------------------+------------------------------+ | sympy_str | 483 ms | 454 ms: 1.07x faster (-6%) | 427 ms: 1.13x faster (-12%) | +-----------------+---------------------------+------------------------------+------------------------------+ xml_etree_generate ------------------ +--------------------+---------------------------+----------------------------+----------------------------+-----------------------------+-----------------------------+ | Benchmark | 2015-04-01 (4fd929b43121) | 2015-07-01 (d7982beca93c) | 2015-10-01 (30b7138fe12b) | 2016-01-01 (899b72cee21c) | 2016-07-01 (355048970b2a) | +====================+===========================+============================+============================+=============================+=============================+ | xml_etree_generate | 282 ms | 267 ms: 1.06x faster (-5%) | 256 ms: 1.10x faster (-9%) | 237 ms: 1.19x faster (-16%) | 212 ms: 1.33x faster (-25%) | +--------------------+---------------------------+----------------------------+----------------------------+-----------------------------+-----------------------------+