멀티 스레드(Multi-Thread ) 응용하기 with Python

블로그를 참고해서 멀티 스레드(Multi-Thread)를 응용하고자 한다.

 

멀티 스레드 응용하기

1. 싱글 스레드를 이용한이미지 다운로드

사용할 이미지는 Corey Schafer의 Python Threading Tutorial에서 가져온 15장 이미지를 사용한다. 주어진 이미지는 고해상도 이미지를 사용하며, 다운로드에 약간 시간이 소요된다. 

import time
import os
import requests
 
img_urls = [
    'https://images.unsplash.com/photo-1516117172878-fd2c41f4a759',
    'https://images.unsplash.com/photo-1532009324734-20a7a5813719',
    'https://images.unsplash.com/photo-1524429656589-6633a470097c',
    'https://images.unsplash.com/photo-1530224264768-7ff8c1789d79',
    'https://images.unsplash.com/photo-1564135624576-c5c88640f235',
    'https://images.unsplash.com/photo-1541698444083-023c97d3f4b6',
    'https://images.unsplash.com/photo-1522364723953-452d3431c267',
    'https://images.unsplash.com/photo-1513938709626-033611b8cc03',
    'https://images.unsplash.com/photo-1507143550189-fed454f93097',
    'https://images.unsplash.com/photo-1493976040374-85c8e12f0c0e',
    'https://images.unsplash.com/photo-1504198453319-5ce911bafcde',
    'https://images.unsplash.com/photo-1530122037265-a5f1f91d3b99',
    'https://images.unsplash.com/photo-1516972810927-80185027ca84',
    'https://images.unsplash.com/photo-1550439062-609e1531270e',
    'https://images.unsplash.com/photo-1549692520-acc6669e2f0c'
]
 
if __name__ == '__main__':
    start = time.perf_counter()
 
    save_file_dir = './images'
    if not os.path.exists(save_file_dir):
        os.makedirs(save_file_dir)
 
    for img_url in img_urls:
        img_bytes = requests.get(img_url).content
        img_name = img_url.split('/')[3]
        img_path = os.path.join(save_file_dir,f'{img_name}.jpg')
        
        with open(img_path, 'wb') as img_file:
            img_file.write(img_bytes)
            print(f'{img_name} was downloaded...')
 
    finish = time.perf_counter()
 
    print(f'Finished in {round(finish-start, 2)} second(s)')

[실행결과] 5.31초가 소요되었으며, 디렉토리에 이미지 폴더가 생긴 것을 확인할 수 있다.

 

 

2. 멀티 스레드를 이용한이미지 다운로드

다음으로는 멀티 스레드를 사용하여 이미지를 다운받아 보자.

import time
import os
import requests
import concurrent.futures
 
img_urls = [
    'https://images.unsplash.com/photo-1516117172878-fd2c41f4a759',
    'https://images.unsplash.com/photo-1532009324734-20a7a5813719',
    'https://images.unsplash.com/photo-1524429656589-6633a470097c',
    'https://images.unsplash.com/photo-1530224264768-7ff8c1789d79',
    'https://images.unsplash.com/photo-1564135624576-c5c88640f235',
    'https://images.unsplash.com/photo-1541698444083-023c97d3f4b6',
    'https://images.unsplash.com/photo-1522364723953-452d3431c267',
    'https://images.unsplash.com/photo-1513938709626-033611b8cc03',
    'https://images.unsplash.com/photo-1507143550189-fed454f93097',
    'https://images.unsplash.com/photo-1493976040374-85c8e12f0c0e',
    'https://images.unsplash.com/photo-1504198453319-5ce911bafcde',
    'https://images.unsplash.com/photo-1530122037265-a5f1f91d3b99',
    'https://images.unsplash.com/photo-1516972810927-80185027ca84',
    'https://images.unsplash.com/photo-1550439062-609e1531270e',
    'https://images.unsplash.com/photo-1549692520-acc6669e2f0c'
]
 
def download_image(input_list):
    img_url = input_list[0]
    save_file_dir =input_list[1]
    img_bytes = requests.get(img_url).content
    img_name = img_url.split('/')[3]
    img_path = os.path.join(save_file_dir,f'{img_name}.jpg')
    
    with open(img_path, 'wb') as img_file:
        img_file.write(img_bytes)
        print(f'{img_name} was downloaded...')
 
if __name__ == '__main__':
    start = time.perf_counter()
 
    save_file_dir = './images'
    if not os.path.exists(save_file_dir):
        os.makedirs(save_file_dir)
 
    with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor:
        executor.map(download_image, [[img_url,save_file_dir] for img_url in img_urls])
 
    finish = time.perf_counter()
 
    print(f'Finished in {round(finish-start, 2)} second(s)')

[실행결과] 3.61초가 소요되었으며, 디렉토리에 이미지 폴더가 생긴 것을 확인할 수 있다. 싱글 스레드 대비 멀티 스레드에서 약 2초가량 빨라진 것을 확인할 수 있다. 각 소요시간은 pc 환경에 따라 상이할 수 있으나 동일한 환경과 조건이라면 멀티 스레드에서 더 빠른 작업속도를 확인할 수 있다.

 

 

출처 : https://zephyrus1111.tistory.com/112?category=835757