Using multiple Python engines (32Bit/64bit and 2.7/3.5)

Python Programming

Question or problem about Python programming:

I would like to use Python for scientific applications and after some research decided that I will use Anaconda as it comes bundled with loads of packages and add new modules using conda install through the cmd is easy.

I prefer to use the 64 bit version for better RAM use and efficiency but
32bit version is needed as well because some libraries are 32bit. Similarly, I prefer to use Python 3.5 as that is the future and the way things go. But loads of libraries are still 2.7 which means I need both.

I have to install 4 versions of Anaconda (64bit 2.7, 64bit 3.5, 32bit 2.7, 64bit 3.5). Each version is about 380MB. I am aiming to use Jupyter notebook and Spyder as the IDE. I had to switch between versions when required. I had conflicting libraries, path issues and all sorts of weird problems.

So, I am planning to do a clean install from scratch. I would like to know if there is a more sensible way to handle this. I use Windows 7 64 bit for now if that matters.

How to solve the problem:

Solution 1:

Make sure to set the right environmental variables (https://github.com/conda/conda/issues/1744)

Create a new environment for 32bit Python 2.7:

set CONDA_FORCE_32BIT=1
conda create -n py27_32 python=2.7

Activate it:

set CONDA_FORCE_32BIT=1
activate py27_32

Deactivate it:

deactivate py27_32

Create one for 64 bit Python 3.5:

set CONDA_FORCE_32BIT=
conda create -n py35_64 python=3.5

Activate it:

set CONDA_FORCE_32BIT=
activate py35_64

The best would be to write the activation commands in a batch file so that you have to type only one command and cannot forget to set the right 32/64 bit flag.

UPDATE

You don’t need to install a full Anaconda distribution for this. Miniconda is enough:


These Miniconda installers contain the conda package manager and Python. Once Miniconda is installed, you can use the conda command to install any other packages and create environments, etc. …
There are two variants of the installer: Miniconda is Python 2 based and Miniconda3 is Python 3 based. Note that the choice of which Miniconda is installed only affects the root environment. Regardless of which version of Miniconda you install, you can still install both Python 2.x and Python 3.x environments.

I would recommend you to use Miniconda3 64-bit as your root environment.

You can always install a full Anaconda later with:

conda install anaconda

Note that it might downgrade some of your previously install packages in your active environment.

Solution 2:

Setting the Subdirectory Constraint

Conda has a configuration variable subdir that can be used to constrain package searching to platforms (e.g., win-32). I think the simplest procedure is to create the empty env, set it’s subdir, then proceed with the (constrained) installations. For example,

win-32, Python 2.7
conda create -n py27_32
conda activate py27_32
conda config --env --set subdir win-32
conda install python=2.7
win-64, Python 3.7
conda create -n py37_64
conda activate py37_64
conda config --env --set subdir win-64
conda install python=3.7

Alternatively, if you need to, for example, create an environment from a YAML file, but want a win-32 platform, one can use the CONDA_SUBDIR environment variable:

set CONDA_SUBDIR=win-32
conda env create -f env.yaml -n my_env_32
set CONDA_SUBDIR=
conda activate my_env_32
conda config --env --set subdir win-32

The nice thing about this procedure is the variable will now always be set whenever activating the env, so future changes to the env will remain within the specified subdirectory.


Ad Hoc Constraints

It is also possible to specify the platform in the --channel|-c argument:

conda install -c defaults/win-32 --override-channels python=3.7

Here the --override-channels is required to ensure that only the provided channel(s) and subdirectory (win-32) is used.

However, setting the subdir on the whole env is likely a more reliable practice.


YAML Constraints

It is also possible to use subdir specifications in a YAML environment definition. For example,

py37_win32.yaml

name: py37_win32
channels:
 - defaults/win-32
dependencies:
 - python=3.7

@Bicudo has tried this and confirms it works, but notes that it does not set any environment-specific constraints on future updates to the environment. Hence, it would be good practice to still set the subdir after environment creation, e.g.,

conda env create -f py37_win32.yaml
conda activate py37_win32
conda config --env --set subdir win-32

Alternatively, beginning with Conda v4.9, one can also specify environment variables as part of the YAML. That is, one can effectively define an environment’s CONDA_SUBDIR value at environment creation:

py37_win32.yaml

name: py37_win32
channels:
 - defaults/win-32
dependencies:
 - python=3.7
variables:
  CONDA_SUBDIR: win-32

Solution 3:

I just wanted to add to Mike Mullers answer, as I also wanted my IPython to switch between 32 bit and 64 bit.

After setting up the 32bit or 64bit environment. Use the following commands

pip install ipykernel

to install ipykernel on this env. Then assign it with:

python -m ipykernel install --user --name myenv --display-name "Python (myenv)"

here myenv is the name of your new environment. See this page here for further details on switching kernels – http://ipython.readthedocs.io/en/stable/install/kernel_install.html

Hope this helps!