In scientific computing, testing means comparing values, which can be numpy
arrays of floats. This is tricky, since rounding errors can cause values to be
“different”, although negligibly. The standard workaround for this is to
rely on np.allclose, which tries to assert whether all values are closer
than a given tolerance. This is great, but is not well captured by pytest
(our recommended
testing package), especially when a difference does indeed arise.
As en example, this is what a failing np.allclose yields in pytest:
> assert np.allclose(np.load(str(datadir / f"{var_}.npy")), locals()[var_])
E AssertionError: assert False
E + where False = <function allclose at 0x10f922a60>(array([0.0186303 , 0.02865569, 0.06371556, 0.22936974, 0.65780499,\n 1.42285828, 2.36019338, 3.34341712, 4.3790955 , 5.32992929]), array([0.01461052, 0.02544779, 0.06163712, 0.21471948, 0.68202262,\n 1.41511093, 2.3613271 , 3.37230873, 4.38335094, 5.36408335]))
E + where <function allclose at 0x10f922a60> = np.allclose
E + and array([0.0186303 , 0.02865569, 0.06371556, 0.22936974, 0.65780499,\n 1.42285828, 2.36019338, 3.34341712, 4.3790955 , 5.32992929]) = <function load at 0x10fb1cc10>('/private/var/folders/kc/q0thl35n2_s371khrfmjmmbh00030h/T/pytest-of-lapeyre/pytest-12/test_tools0/MI_nonlin.npy')
E + where <function load at 0x10fb1cc10> = np.load
E + and '/private/var/folders/kc/q0thl35n2_s371khrfmjmmbh00030h/T/pytest-of-lapeyre/pytest-12/test_tools0/MI_nonlin.npy' = str((local('/private/var/folders/kc/q0thl35n2_s371khrfmjmmbh00030h/T/pytest-of-lapeyre/pytest-12/test_tools0') / 'MI_nonlin.npy'))
A neat little package called pytest-allclose does this a lot better. The same failing test as before yields:
allclose first 5 failures:
(0,): 0.01708240412706181 0.017734838901692883
(1,): 0.02186258822666283 0.023818469278532636
(2,): 0.06113776747072141 0.07079141825370325
(3,): 0.2189457580907803 0.23368660505997008
(4,): 0.6798632188844984 0.6736695124904171
That’s more like it! So how do you get this result? It’s a simple 2 step process:
pip install pytest-allclose- In your test file, use the
allclosefixturedef test_feature(allclose): assert allclose(test_array, target_array)
No more excuses not to make great tests for scientific results now!