[Airflow, Mysql] UnicodeEncodeError: ‘latin-1’ codec can’t encode characters in position 652–653: ordinal not in range(256
현상
- Airflow에서 Mysql-> Postgresql의 Task 내 아래 오류 발생
[2020–10–27 09:08:44,144] {taskinstance.py:1150} ERROR — ‘latin-1’ codec can’t encode characters in position 652–653: ordinal not in range(256)
Traceback (most recent call last):
File “/usr/local/lib/python3.6/site-packages/airflow/models/taskinstance.py”, line 984, in _run_raw_task
result = task_copy.execute(context=context)
File “/root/airflow/plugins/operators/MySqlToPostgresOperator.py”, line 50, in execute
cursor.execute(self.sql, self.params)
File “/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py”, line 248, in execute
query = query.encode(db.encoding, ‘surrogateescape’)
UnicodeEncodeError: ‘latin-1’ codec can’t encode characters in position 652–653: ordinal not in range(256)
2. Mysql의 언어셋: utf-8
원인
Airflow 단에서 mysql Connection의 charset을 기본적으로 latin1으로 인코드하는 것으로 추정
조치
아래와 같이 utf8로 지정함 -해결
