Split many ArcGIS rasters in parallel using Python multiprocessing -
i looking split 10 images 2 parts each (20 resulting images). images 4-band (r,g,b,nir) naip imagery available this website. using arcpy
package arcgis split 1 image @ time:
import arcpy, os inws = r'd:\temp\temp_naip' #contains ~10 .tif images outws = r'd:\temp\temp_naip_tiles' arcpy.env.workspace = inws rasters = arcpy.listrasters() ras in rasters: arcpy.splitraster_management( ras, outws, os.path.basename(ras).split('.')[0], split_method='number_of_tiles', format='tiff', num_rasters='1 2', overlap=50, units='pixels)
how can integrate multiprocessing
module above script process, say, 4 images @ time?
btw, aware of blog post combines multiprocessing
, arcpy
, although examples specific vector data , cannot figure out how utilize tools process imagery.
barring resource sharing issues, converting simple for-loop multiprocessing easy multiprocessing.pool
. try this:
from multiprocessing import pool import arcpy, os inws = r'd:\temp\temp_naip' #contains ~10 .tif images outws = r'd:\temp\temp_naip_tiles' arcpy.env.workspace = inws rasters = arcpy.listrasters() def process_img(ras): arcpy.splitraster_management( ras, outws, os.path.basename(ras).split('.')[0], split_method='number_of_tiles', format='tiff', num_rasters='1 2', overlap=50, units='pixels') pool = pool(processes=4) pool.map(process_img, rasters)
so long rasters
iterable, should mappable process pool. keep in mind each process "inherit" parent process' stack, such each process use it's own copy of arcpy.env.workspace
.