使用移动的内核窗口时,Numba失败:TypingError,定义错误

我正在从事一个项目,该项目需要我构建一系列使用移动内核窗口来操纵存储在矩阵中的高程数据的功能。

我最初关于如何优化两个嵌套for循环的问题在此处link中得到了回答。该解决方案涉及将我的代码与 Numba 并行化。当我尝试将另一个移动的内核窗口函数改编为numba时,遇到了numpy.gradient的问题,目前numba不支持。因此,我将我的功能分为两部分: A)不在Numba中的预处理功能和 B)用Numba编写的主要功能。我现在正在尝试使用numba运行该功能以使其正常工作。我继续遇到错误(如下所示),经过数周的搜索堆栈交换和互联网的解决方案后,现在我感到非常沮丧。

功能如下所示:

first function

def DCE_preprocess(DEM,cellsize,w):

    [nrows,ncols] = np.shape(DEM)

    #initiate an empty array same size as dem
    rms = DEM*np.nan
#    rms = np.float32(rms)

#    #compute the directional cosines
    [fx,fy] = np.gradient(DEM,cellsize)


    grad = np.sqrt(fx**2 + fy**2)
    asp = np.arctan2(fy,fx)



    grad=np.pi/2-np.arctan(grad) #normal of steepest slope
    asp[asp<np.pi]=asp[asp<np.pi]+[np.pi/2]
    asp[asp<0]=asp[asp<0]+[2*np.pi]

    #spherical to cartesian conversion
    r = 1
    cy = r * np.cos(grad) * np.sin(asp)
    cx = r * np.cos(grad) * np.cos(asp)
    cz = r * np.sin(grad)

    return(cx,cy,cz)

返回第二个函数的输入: second function

eps = np.finfo(float).eps

@nb.njit(parallel=True)
def DC_eig_par(DEM,w,cx,cz,eps):
    [nrows,ncols] = np.shape(DEM)
#
#    #initiate an empty array same size as dem
    rms = DEM*np.nan
    rms.astype(np.float32)

    #cycling through the DEM
    nw=(w*2)**2

    for i in nb.prange(w+1,nrows-w):


        for j in range(w+1,(ncols-w)):

            d1=nb.int32(np.linspace(i-w,i+w,11))
            d2=nb.int32(np.linspace(j-w,j+w,11))

            tempx = cx[(d1[0]):(d1[-1]),(d2[0]):(d2[-1])]
            tx=np.reshape(tempx,-1)


            tempy = cy[(d1[0]):(d1[-1]),(d2[0]):(d2[-1])]
            ty=np.reshape(tempy,-1)

            tempz = np.empty([10,10],dtype = np.float32)
            tempz = cz[(d1[0]):(d1[-1]),(d2[0]):(d2[-1])]
            tz=np.reshape(tempz,-1)

            if np.max(np.isnan(np.concatenate((tx,ty,tz)))) == 0:
                T=np.array([[np.sum(tx**2),np.sum(tx*ty),np.sum(tx*tz)],[np.sum(ty*tx),np.sum(ty**2),np.sum(ty*tz)],[np.sum(tz*tx),np.sum(tz*ty),np.sum(tz**2)]])


                [Te,_] = np.linalg.eig(T) # this step is a bit different from the matlab version b/c np.eig outputs two values.
                l = (Te/nw)
                l[l<eps] = 0
                rms[i,j] = 1/np.log(l[0]/l[1])


            else:
                rms[i,j] = np.nan

    return(rms)

第二个功能现在给我以下错误:

Traceback (most recent call last):

  File "<ipython-input-65-af12df87ecaa>",line 8,in <module>
    rp = DC_eig_par(DEM,eps)

  File "C:\programdata\Anaconda3\lib\site-packages\numba\dispatcher.py",line 351,in _compile_for_args
    error_rewrite(e,'typing')

  File "C:\programdata\Anaconda3\lib\site-packages\numba\dispatcher.py",line 318,in error_rewrite
    reraise(type(e),e,None)

  File "C:\programdata\Anaconda3\lib\site-packages\numba\six.py",line 658,in reraise
    raise value.with_traceback(tb)

TypingError: Invalid use of Function(<built-in function getitem>) with argument(s) of type(s): (int32,Literal[int](0))
 * parameterized
In definition 0:
    All templates rejected with literals.
In definition 1:
    All templates rejected without literals.
In definition 2:
    All templates rejected with literals.
In definition 3:
    All templates rejected without literals.
In definition 4:
    All templates rejected with literals.
In definition 5:
    All templates rejected without literals.
In definition 6:
    All templates rejected with literals.
In definition 7:
    All templates rejected without literals.
In definition 8:
    All templates rejected with literals.
In definition 9:
    All templates rejected without literals.
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: typing of intrinsic-call at U:\GHP\Projects\NSF - Morris Landslides\Code\Developmemnt\Wavelets\DC_eig_par.py (40)
[2] During: typing of static-get-item at U:\GHP\Projects\NSF - Morris Landslides\Code\Developmemnt\Wavelets\DC_eig_par.py (40)

代码在40tempx = cx[(d1[0]):(d1[-1]),(d2[0]):(d2[-1])]上失败。我不确定发生了什么。我已经确定所有变量在整个过程中都是一致的类型,或者我想。非常感谢任何人提供的有关解决此问题的方法的见解或帮助。

并行运行相似的功能会将1165 x 1355矩阵的6分钟操作缩减为几秒钟。我希望此功能具有类似的加速效果。 非常感谢

lsy94522 回答:使用移动的内核窗口时,Numba失败:TypingError,定义错误

经过大量的试验和错误,并深入研究了TraceBack,我能够成功解决此问题。我的新功能如下:

@nb.njit(parallel=True)
def DC_eig_par(DEM,w,cx,cy,cz,eps):
    [nrows,ncols] = np.shape(DEM)
#
#    #initiate an empty array same size as dem
    rms = DEM*np.nan
    rms.astype(np.float32)

    #Compute RMS cycling through the DEM
    nw=(w*2)**2

    for i in nb.prange(w+1,nrows-w):


        for j in range(w+1,(ncols-w)):
#            d1=np.int16(np.linspace(i-w,i+w,11))
#            d2=np.int16(np.linspace(j-w,j+w,11))

            tempx = cx[i-w:i+w,j-w:j+w]
            tx = tempx.flatten()



            tempy = cy[i-w:i+w,j-w:j+w]
            ty = tempy.flatten()


            tempz = cz[i-w:i+w,j-w:j+w]
            tz = tempz.flatten()



            if (np.isnan(np.concatenate((tx,ty,tz)))).sum() == 0:
                T=np.array([[np.sum(tx**2),np.sum(tx*ty),np.sum(tx*tz)],[np.sum(ty*tx),np.sum(ty**2),np.sum(ty*tz)],[np.sum(tz*tx),np.sum(tz*ty),np.sum(tz**2)]])


                [Te,_] = np.linalg.eig(T) # this step is a bit different from the matlab version b/c np.eig outputs two values.
                l = (Te/nw)
                l[l<eps] = 0
                rms[i,j] = 1/np.log(l[0]/l[1])


            else:
                rms[i,j] = np.nan

    return(rms)

主要变化是我:

  1. 取消了对reshape的呼叫,而是使用了tempx.flatten
  2. if语句中对max的调用已更改为.sum()

现在,代码就像一个符咒一样运行。非并行代码运行所需的时间比6分钟要花费6秒。

本文链接:https://www.f2er.com/3062195.html

大家都在问