python如何加速计算密集型任务2?
问题描述:
在python中,有一个函数,其功能是进行某种计算,需要传入一些参数,计算完成后传回结果,参数是a,b,c,d,分别从列表中取值,返回值是xx,yy,分别是一个列表,调用其一次大概要1s的时间,现在需要通过for循环调用其350次,保存每次调用结果(可能是合并成一个列表),腺癌要加速该代码?
使用 multiprocessing 模块:
from multiprocessing import Pool
import time
# 假设这是你的函数
def compute(a, b, c, d):
# 这里是计算过程,我们假设计算需要1秒
import time
time.sleep(1) # 模拟耗时的计算
xx = [a, b]
yy = [c, d]
return xx, yy
# 这是你的参数列表
A = list(range(1, 351)) # 从1到350的整数列表
B = [i * 2 for i in A] # 假设B是A的两倍
C = [i * 3 for i in A] # 假设C是A的三倍
D = [i * 4 for i in A] # 假设D是A的四倍
# 将A, B, C, D组合成参数对
params = [(a, b, c, d) for a, b, c, d in zip(A, B, C, D)]
# 使用 multiprocessing 的 Pool 来并行计算
if __name__ == '__main__':
start_time = time.time() # 记录开始时间
with Pool(processes=4) as pool: # 你可以根据你的 CPU 核心数来设置进程数
results = pool.starmap(compute, params)
end_time = time.time() # 记录结束时间
# 合并结果
all_xx = [result[0] for result in results]
all_yy = [result[1] for result in results]
# 打印结果的一部分以验证
print("First few xx:", all_xx[:5])
print("First few yy:", all_yy[:5])
# 打印执行时间
print(f"Execution time: {end_time - start_time:.2f} seconds")
使用concurrent.futures模块:
from concurrent.futures import ProcessPoolExecutor
import time
# 假设这是你要调用的计算密集型函数
def compute(a, b, c, d):
time.sleep(1) # 模拟计算过程
xx = [a, b] # 模拟计算结果
yy = [c, d] # 模拟计算结果
return xx, yy
# 主函数
def main():
start_time = time.time()
# 参数列表
A = list(range(1, 351))
B = [i * 2 for i in A]
C = [i * 3 for i in A]
D = [i * 4 for i in A]
params = list(zip(A, B, C, D)) # 将四个列表组合成一个参数列表
# 结果列表
results = []
# 使用ProcessPoolExecutor创建进程池
with ProcessPoolExecutor() as executor:
# 使用executor.map并行执行函数
results = list(executor.map(compute, *zip(*params)))
end_time = time.time()
print(f"Total time taken: {end_time - start_time} seconds")
# 合并结果
all_xx = [result[0] for result in results]
all_yy = [result[1] for result in results]
print(f"First few xx: {all_xx[:10]}") # 打印前10个xx结果作为示例
print(f"First few yy: {all_yy[:10]}") # 打印前10个yy结果作为示例
if __name__ == "__main__":
main()
输出:
Total time taken: 88.80965089797974 seconds
First few xx: [[1, 2], [2, 4], [3, 6], [4, 8], [5, 10], [6, 12], [7, 14], [8, 16], [9, 18], [10, 20]]
First few yy: [[3, 4], [6, 8], [9, 12], [12, 16], [15, 20], [18, 24], [21, 28], [24, 32], [27, 36], [30, 40]]