AnaConda环境下安装librosa包超时
问题重现
大家在AnaConda环境下安装包的过程中肯定会遇到下载缓慢的问题。
超时信息如下(为了大家方便通过超时信息找到,牺牲了一些页面体验,把异常信息贴出来):
conda.exceptions.CondaRuntimeError: Runtime error: Could not open ‘/home/ubuntu/anaconda3/pkgs/icu-64.2-he1b5a44_1.tar.bz2.part‘ for writing (HTTPSConnectionPool(host=‘conda.anaconda.org‘, port=443): Read timed out.). During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/exceptions.py", line 473, in conda_exception_handler return_value = func(*args, **kwargs) File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/cli/main.py", line 144, in _main exit_code = args.func(args, p) File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/cli/main_install.py", line 80, in execute install(args, parser, ‘install‘) File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/cli/install.py", line 422, in install raise CondaSystemExit(‘Exiting‘, e) File "/home/ubuntu/anaconda3/lib/python3.5/contextlib.py", line 77, in __exit__ self.gen.throw(type, value, traceback) File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/cli/common.py", line 573, in json_progress_bars yield File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/cli/install.py", line 420, in install raise CondaRuntimeError(‘RuntimeError: %s‘ % e) conda.exceptions.CondaRuntimeError: Runtime error: RuntimeError: Runtime error: Could not open ‘/home/ubuntu/anaconda3/pkgs/icu-64.2-he1b5a44_1.tar.bz2.part‘ for writing (HTTPSConnectionPool(host=‘conda.anaconda.org‘, port=443): Read timed out.).
或者如下:
Fetching package metadata ...INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.ustc.edu.cn INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.ustc.edu.cn INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): repo.continuum.io INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.ustc.edu.cn INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.ustc.edu.cn INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): repo.continuum.io INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): conda.anaconda.org INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): conda.anaconda.org INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn .INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): repo.continuum.io .INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): repo.continuum.io .........Could not connect to https://conda.anaconda.org/conda-forge/linux-64/ INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): conda.anaconda.org ..INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.ustc.edu.cn INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn ...INFO requests.packages.urllib3.connectionpool:_get_conn(249): Resetting dropped connection: mirrors.ustc.edu.cn ...INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): repo.continuum.io WARNING requests.packages.urllib3.connectionpool:urlopen(664): Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by ‘ReadTimeoutError("HTTPSConnectionPool(host=‘repo.continuum.io‘, port=443): Read timed out. (read timeout=3.05)",)‘: /pkgs/free/linux-64/repodata.json.bz2 INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (2): repo.continuum.io
解决方法
通常的解决方法是切换到国内的镜像源
如conda切换到中科大源,命令行下执行
conda config --add channels https://mirrors.ustc.edu.cn/anaconda/pkgs/main/ conda config --add channels https://mirrors.ustc.edu.cn/anaconda/pkgs/free/ conda config --add channels https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/
pip切换源参考这里:pip设置安装源
但是有时候即便切换了源,有些页面因为网络或者服务器原因下载仍旧是缓慢,直至出现超时。
笔者最近下载librosa,执行以下命令,最终超时下载不了。
conda install librosa -c conda-forge
后来又添加了一个-v参数查看执行命令情况
conda install librosa -v -c conda-forge
发现有执行超时的情况
后来在stackoverflow上面找到了答案,AnaConda在4.3版本以下读取超时都是被硬编码成60秒,这个显然是不合理的。
I had the same issue. In conda < 4.3.0, the timeout was hardcoded to 60 seconds. In Windows, I‘m guessing, that‘s not enough for huge package with a lot of binary
files like qt since most likely the virus scanner kicks in to check the files. You can hack your ~userid\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda\fetch.py
and change the hard-coded constant yourself, eg. from 60 to 300.
With newer version (I‘m not sure when this was added, but I see this in my conda 4.3.22), running conda config --show
gave: remote_read_timeout_secs: 60.0
.
So I believe you can modify it by adding it to your ~userid\.condarc
file.
笔者本地的是4.2.0版本Anaconda3-4.2.0-Linux-x86_64.sh
尝试执行命令报错:
conda config --set remote_read_timeout_secs 3600
报错信息:
:~$ conda config --set remote_read_timeout_secs 3600 CondaValueError: Value error: Error key must be one of add_binstar_token, update_dependencies, binstar_upload, always_copy, anaconda_upload, add_pip_as_python_dependency, allow_other_channels, channel_priority, changeps1, always_yes, shortcuts, channel_alias, use_pip, auto_update_conda, add_anaconda_token, ssl_verify, client_cert_key, client_cert, show_channel_urls, offline, allow_softlinks, not remote_read_timeout_secs
也就是当前版本4.2.0还不支持这个remote_read_timeout_secs环境变量设置
笔者在最新版的官方文档上找到超时相关资料,各位可以根据实际情况酌情使用这两个参数:
远程连接超时(默认9.15秒)
# # remote_connect_timeout_secs (float) # # The number seconds conda will wait for your client to establish a # # connection to a remote url resource. # # # remote_connect_timeout_secs: 9.15
远程读取超时(默认60秒)
# # remote_read_timeout_secs (float) # # Once conda has connected to a remote resource and sent an HTTP # # request, the read timeout is the number of seconds conda will wait for # # the server to send a response. # # # remote_read_timeout_secs: 60.0
结论
升级AnaConda版本或者更换镜像源
wget https://repo.anaconda.com/archive/Anaconda3-2019.10-Linux-x86_64.sh chmod +x Anaconda3-2019.10-Linux-x86_64.sh
安装过程很简单,一路回车即可
配置环境变量
vim ~/.bashrc
# added by Anaconda3 4.2.0 installer export PATH="/home/ubuntu/anaconda3/bin:$PATH" export CUDA_HOME=/usr/local/cuda-10.2 export PATH="$CUDA_HOME/bin:$PATH" export LD_LIBRARY_PATH="$CUDA_HOME/lib64:$LD_LIBRARY_PATH"
参考来源:https://stackoverflow.com/questions/42797957/changing-timeout-limit-when-using-conda-install
https://docs.conda.io/projects/conda/en/latest/configuration.html