我正在从事一个项目,该项目需要我构建一系列使用移动内核窗口来操纵存储在矩阵中的高程数据的功能。
我最初关于如何优化两个嵌套for循环的问题在此处link中得到了回答。该解决方案涉及将我的代码与 Numba 并行化。当我尝试将另一个移动的内核窗口函数改编为numba时,遇到了numpy.gradient
的问题,目前numba不支持。因此,我将我的功能分为两部分: A)不在Numba中的预处理功能和 B)用Numba编写的主要功能。我现在正在尝试使用numba运行该功能以使其正常工作。我继续遇到错误(如下所示),经过数周的搜索堆栈交换和互联网的解决方案后,现在我感到非常沮丧。
功能如下所示:
first function
def DCE_preprocess(DEM,cellsize,w):
[nrows,ncols] = np.shape(DEM)
#initiate an empty array same size as dem
rms = DEM*np.nan
# rms = np.float32(rms)
# #compute the directional cosines
[fx,fy] = np.gradient(DEM,cellsize)
grad = np.sqrt(fx**2 + fy**2)
asp = np.arctan2(fy,fx)
grad=np.pi/2-np.arctan(grad) #normal of steepest slope
asp[asp<np.pi]=asp[asp<np.pi]+[np.pi/2]
asp[asp<0]=asp[asp<0]+[2*np.pi]
#spherical to cartesian conversion
r = 1
cy = r * np.cos(grad) * np.sin(asp)
cx = r * np.cos(grad) * np.cos(asp)
cz = r * np.sin(grad)
return(cx,cy,cz)
返回第二个函数的输入:
second function
eps = np.finfo(float).eps
@nb.njit(parallel=True)
def DC_eig_par(DEM,w,cx,cz,eps):
[nrows,ncols] = np.shape(DEM)
#
# #initiate an empty array same size as dem
rms = DEM*np.nan
rms.astype(np.float32)
#cycling through the DEM
nw=(w*2)**2
for i in nb.prange(w+1,nrows-w):
for j in range(w+1,(ncols-w)):
d1=nb.int32(np.linspace(i-w,i+w,11))
d2=nb.int32(np.linspace(j-w,j+w,11))
tempx = cx[(d1[0]):(d1[-1]),(d2[0]):(d2[-1])]
tx=np.reshape(tempx,-1)
tempy = cy[(d1[0]):(d1[-1]),(d2[0]):(d2[-1])]
ty=np.reshape(tempy,-1)
tempz = np.empty([10,10],dtype = np.float32)
tempz = cz[(d1[0]):(d1[-1]),(d2[0]):(d2[-1])]
tz=np.reshape(tempz,-1)
if np.max(np.isnan(np.concatenate((tx,ty,tz)))) == 0:
T=np.array([[np.sum(tx**2),np.sum(tx*ty),np.sum(tx*tz)],[np.sum(ty*tx),np.sum(ty**2),np.sum(ty*tz)],[np.sum(tz*tx),np.sum(tz*ty),np.sum(tz**2)]])
[Te,_] = np.linalg.eig(T) # this step is a bit different from the matlab version b/c np.eig outputs two values.
l = (Te/nw)
l[l<eps] = 0
rms[i,j] = 1/np.log(l[0]/l[1])
else:
rms[i,j] = np.nan
return(rms)
第二个功能现在给我以下错误:
Traceback (most recent call last):
File "<ipython-input-65-af12df87ecaa>",line 8,in <module>
rp = DC_eig_par(DEM,eps)
File "C:\programdata\Anaconda3\lib\site-packages\numba\dispatcher.py",line 351,in _compile_for_args
error_rewrite(e,'typing')
File "C:\programdata\Anaconda3\lib\site-packages\numba\dispatcher.py",line 318,in error_rewrite
reraise(type(e),e,None)
File "C:\programdata\Anaconda3\lib\site-packages\numba\six.py",line 658,in reraise
raise value.with_traceback(tb)
TypingError: Invalid use of Function(<built-in function getitem>) with argument(s) of type(s): (int32,Literal[int](0))
* parameterized
In definition 0:
All templates rejected with literals.
In definition 1:
All templates rejected without literals.
In definition 2:
All templates rejected with literals.
In definition 3:
All templates rejected without literals.
In definition 4:
All templates rejected with literals.
In definition 5:
All templates rejected without literals.
In definition 6:
All templates rejected with literals.
In definition 7:
All templates rejected without literals.
In definition 8:
All templates rejected with literals.
In definition 9:
All templates rejected without literals.
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: typing of intrinsic-call at U:\GHP\Projects\NSF - Morris Landslides\Code\Developmemnt\Wavelets\DC_eig_par.py (40)
[2] During: typing of static-get-item at U:\GHP\Projects\NSF - Morris Landslides\Code\Developmemnt\Wavelets\DC_eig_par.py (40)
代码在40
行tempx = cx[(d1[0]):(d1[-1]),(d2[0]):(d2[-1])]
上失败。我不确定发生了什么。我已经确定所有变量在整个过程中都是一致的类型,或者我想。非常感谢任何人提供的有关解决此问题的方法的见解或帮助。
并行运行相似的功能会将1165 x 1355矩阵的6分钟操作缩减为几秒钟。我希望此功能具有类似的加速效果。 非常感谢