They found two quick fixes - number 1 and number 2 - which can be used to solve the issue with Homebrew Clang and native Apple Clang, respectively. According to this discussion on oneDNN, the problem seems to be that FindOpenMP.cmake does not work properly with Apple Clang. Rather than use native threading, there is an easy fix to make OpenMP work properly as per Linux and Windows binaries. In an equivalent linux binary, I see that mkl_get_max_threads() matches at::get_num_threads(), the expected behavior. It's unclear what the performance implications are, but if MKL-based BLAS and LAPACK ops are unable to benefit from parallelism, this would certainly be a concern. by linking the library and include directories from my conda environment), I see that mkl_get_max_threads() = 6.

When I check the same mkl distribution in C++ (e.g. An unintended consequence for Mac users is that, once you have called a Timer with the default num_threads=1, parallelism will now be disabled for the remainder of your notebook/script (especially concerning in notebooks, since the above warning message is printed to terminal and thus the user doesn't know this happened).Īnother strange finding with the Mac OS binaries is that, looking at torch._config_.parallel_info(), pytorch seems to think my mkl distribution is configured with mkl_get_max_threads() : 1, even though it reports at::get_num_threads() : 6. I also can't use the 'num_threads' feature in the new tool to test runtime performance at different thread counts. This is a bummer, since I can't tune the number of threads for an op as per the PyTorch threading tutorial. Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)

Parallel mac os