urllib request.py urlopen SSL 인증서 에러 문제 해결 SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED]

URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)>

PyTorch에서 ResNet 신경망을 불러오다가 접한 에러이다.

회사에서 작업을 하다보면, SSL인증서 문제로 url접근에 문제가 되는 경우가 이전에도 있었다.

2020/04/14 - [IT-Engineering] - git push -u origin master 실행시 SSL certificate problem: self signed certificate in certificate chain 에러 발생 해결

git push -u origin master 실행시 SSL certificate problem: self signed certificate in certificate chain 에러 발생 해결

맨 아래의 로그가 전체로그이다.

결론적으로, urlopen함수를 호출하면서 생긴 에러였다.

이 경우는 PyTorch torch 패키지의 hub.py에서 download_url_to_file을 호출하면서 발생한 에러였다.

설치된 패키지 코드를 건드린다는 것이 찜찜하기는 하지만 간단하게 수정을 가할 수 있다.

{conda python경로}/site-packages/torch/hub.py 코드를 수정했다.

해당 코드는 이런 형태였는데

u = urlopen(url)

아래와 같이 적당한 위치에 ssl패키지를 import해주고

import ssl

위의 urlopen코드를 아래와 같이 수정해주면, ssl인증 문제를 넘길 수 있다.

context = ssl._create_unverified_context()
u = urlopen(url,context=context)

더 좋은 방법이 있으면 공유해주시면 감사하겠다.

python 패키지들의 함수가 ssl인증서를 필요로 할 경우 매번 이렇게 수정해주어야 할려나 싶다 ㅠㅠ

---------------------------------------------------------------------------
SSLCertVerificationError                  Traceback (most recent call last)
~/anaconda3/envs/ai/lib/python3.7/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1318                 h.request(req.get_method(), req.selector, req.data, headers,
-> 1319                           encode_chunked=req.has_header('Transfer-encoding'))
   1320             except OSError as err: # timeout error

~/anaconda3/envs/ai/lib/python3.7/http/client.py in request(self, method, url, body, headers, encode_chunked)
   1251         """Send a complete request to the server."""
-> 1252         self._send_request(method, url, body, headers, encode_chunked)
   1253 

~/anaconda3/envs/ai/lib/python3.7/http/client.py in _send_request(self, method, url, body, headers, encode_chunked)
   1297             body = _encode(body, 'body')
-> 1298         self.endheaders(body, encode_chunked=encode_chunked)
   1299 

~/anaconda3/envs/ai/lib/python3.7/http/client.py in endheaders(self, message_body, encode_chunked)
   1246             raise CannotSendHeader()
-> 1247         self._send_output(message_body, encode_chunked=encode_chunked)
   1248 

~/anaconda3/envs/ai/lib/python3.7/http/client.py in _send_output(self, message_body, encode_chunked)
   1025         del self._buffer[:]
-> 1026         self.send(msg)
   1027 

~/anaconda3/envs/ai/lib/python3.7/http/client.py in send(self, data)
    965             if self.auto_open:
--> 966                 self.connect()
    967             else:

~/anaconda3/envs/ai/lib/python3.7/http/client.py in connect(self)
   1421             self.sock = self._context.wrap_socket(self.sock,
-> 1422                                                   server_hostname=server_hostname)
   1423 

~/anaconda3/envs/ai/lib/python3.7/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    422             context=self,
--> 423             session=session
    424         )

~/anaconda3/envs/ai/lib/python3.7/ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
    869                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
--> 870                     self.do_handshake()
    871             except (OSError, ValueError):

~/anaconda3/envs/ai/lib/python3.7/ssl.py in do_handshake(self, block)
   1138                 self.settimeout(None)
-> 1139             self._sslobj.do_handshake()
   1140         finally:

SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)

During handling of the above exception, another exception occurred:

URLError                                  Traceback (most recent call last)
<ipython-input-3-489dfb92ded8> in <module>
----> 1 model_ft = resnet18(pretrained=True)

<ipython-input-1-55f0ae24cc34> in resnet18(pretrained, progress, **kwargs)
    239     """
    240     return _resnet('resnet18', BasicBlock, [2, 2, 2, 2], pretrained, progress,
--> 241                    **kwargs)
    242 
    243 

<ipython-input-1-55f0ae24cc34> in _resnet(arch, block, layers, pretrained, progress, **kwargs)
    225     if pretrained:
    226         state_dict = load_state_dict_from_url(model_urls[arch],
--> 227                                               progress=progress)
    228         model.load_state_dict(state_dict)
    229     return model

~/anaconda3/envs/ai/lib/python3.7/site-packages/torch/hub.py in load_state_dict_from_url(url, model_dir, map_location, progress, check_hash)
    497         sys.stderr.write('Downloading: "{}" to {}\n'.format(url, cached_file))
    498         hash_prefix = HASH_REGEX.search(filename).group(1) if check_hash else None
--> 499         download_url_to_file(url, cached_file, hash_prefix, progress=progress)
    500 
    501     # Note: extractall() defaults to overwrite file if exists. No need to clean up beforehand.

~/anaconda3/envs/ai/lib/python3.7/site-packages/torch/hub.py in download_url_to_file(url, dst, hash_prefix, progress)
    396     #context = ssl._create_unverified_context()
    397     #u = urlopen(url,context=context)
--> 398     u = urlopen(url)
    399     meta = u.info()
    400     if hasattr(meta, 'getheaders'):

~/anaconda3/envs/ai/lib/python3.7/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    220     else:
    221         opener = _opener
--> 222     return opener.open(url, data, timeout)
    223 
    224 def install_opener(opener):

~/anaconda3/envs/ai/lib/python3.7/urllib/request.py in open(self, fullurl, data, timeout)
    523             req = meth(req)
    524 
--> 525         response = self._open(req, data)
    526 
    527         # post-process response

~/anaconda3/envs/ai/lib/python3.7/urllib/request.py in _open(self, req, data)
    541         protocol = req.type
    542         result = self._call_chain(self.handle_open, protocol, protocol +
--> 543                                   '_open', req)
    544         if result:
    545             return result

~/anaconda3/envs/ai/lib/python3.7/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    501         for handler in handlers:
    502             func = getattr(handler, meth_name)
--> 503             result = func(*args)
    504             if result is not None:
    505                 return result

~/anaconda3/envs/ai/lib/python3.7/urllib/request.py in https_open(self, req)
   1360         def https_open(self, req):
   1361             return self.do_open(http.client.HTTPSConnection, req,
-> 1362                 context=self._context, check_hostname=self._check_hostname)
   1363 
   1364         https_request = AbstractHTTPHandler.do_request_

~/anaconda3/envs/ai/lib/python3.7/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1319                           encode_chunked=req.has_header('Transfer-encoding'))
   1320             except OSError as err: # timeout error
-> 1321                 raise URLError(err)
   1322             r = h.getresponse()
   1323         except:

URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)>