Better ways to do things under Python 3 #12735

Following the drop of Python 2 support in the next release (#11881) and specifically this comment, I am listing some new Python 3 features that can replace some old ones.

New in 3.4

• use the pathlib module for dealing with paths. see PEP428

New in 3.5

• replace os.listdir with the much faster os.scandir. see PEP471

New in 3.6

• secrets module to replace os.urandom. see PEP 506

• f-strings instead of str.format. see PEP 498

Thanks @mcdallas - feel free to open PRs if you think changing individual python modules to use those new features is a genuine improvement (rather than just changing to the new thing because it's the new thing).

I don't think we need an issue to track this - just PRs for the individual changes.

@jnewbery I can send some PRs, just want to make sure what is the minimum version you are looking to support? For example ubuntu 16.04 ships with Python 3.5.2 which doesn't support f-strings.

I think we need to support 3.4 at a minimum. See #9485 (comment).

Agree on what @jnewbery said.

New in 3.4

I think we'd need to support python 3.4 until at least April 2019 (EOL for trusty), so the only thing we could make use of is the pathlib module. Generally, just refactoring for the sake of refactoring is discouraged. Though, I wouldn't object changing to the pathlib module when you have to touch the file for other reasons or when using the pathlib module gives large simplifications.

New in 3.5

Using a method that is faster is fine, when it comes "for free". However, since we'd have to support python 3.5 for a while, I don't think we can do that anytime soon. Also, wherever this is used, the bottleneck is not in this method call, so probably not worth trying to optimize.

Closing.

Pull requests welcome, but no need to keep an issue open about purely refactoring related code changes.