UFuncTypeError：ufunc 'clip' 不包含具有签名匹配类型的循环（dtype（' <u32’), dtype(‘<u32’), dtype(‘<u32’)) -> dtype(' <u32’)< div><div id="text_translate"><p> 我使用 Deep Pavlov 框架与 Bert 分类器一起工作，只是因为我需要预测人员的语言是俄语。 基本上，我正在尝试解决多类分类问题。 根据 Deep Pavlov，我们可以轻松地更改配置文件上的一些配置。 我拿了这个配置文件<a href="https://github.com/deepmipt/DeepPavlov/blob/master/deeppavlov/configs/classifiers/rusentiment_convers_bert.json" rel="nofollow noreferrer">https://github.com/deepmipt/DeepPavlov/blob/master/deeppavlov/configs/classifiers/rusentiment_convers_bert.json</a>并训练它，结果我花了大约 13 个小时才完成它我的 model 过拟合。</p><p> 我做了一些改变，尤其是这些：</p><pre> "weight_decay_rate": 0.001, "learning_rate_drop_patience": 1, "learning_rate_drop_div": 2.0, "load_before_drop": True, "min_learning_rate": 1e-03, "attention_probs_keep_prob": 0.5, "hidden_keep_prob": 0.5,</pre><p> 另外，我增加了批量大小，之前是 16：</p><pre> "batch_size": 32</pre><p> 并添加了一些指标：</p><pre> "log_loss", "matthews_correlation",</pre><p> 还将validation_patience更改为1并添加了tensorboard func</p><pre> "validation_patience": 1, "tensorboard_log_dir": "logs/",</pre><p> 就是这样。 这些是我对 model 所做的所有更改，当我尝试训练我的 model 时，它给了我以下错误：</p><pre> UFuncTypeError Traceback (most recent call last) /usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py in _wrapfunc(obj, method, *args, **kwds) 60 try: ---&gt; 61 return bound(*args, **kwds) 62 except TypeError: 15 frames UFuncTypeError: ufunc 'clip' did not contain a loop with signature matching types (dtype('&lt;U32'), dtype('&lt;U32'), dtype('&lt;U32')) -&gt; dtype('&lt;U32') During handling of the above exception, another exception occurred: UFuncTypeError Traceback (most recent call last) &lt;__array_function__ internals&gt; in clip(*args, **kwargs) /usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py in _clip_dep_invoke_with_casting(ufunc, out, casting, *args, **kwargs) 83 # try to deal with broken casting rules 84 try: ---&gt; 85 return ufunc(*args, out=out, **kwargs) 86 except _exceptions._UFuncOutputCastingError as e: 87 # Numpy 1.17.0, 2019-02-24 UFuncTypeError: ufunc 'clip' did not contain a loop with signature matching types (dtype('&lt;U32'), dtype('&lt;U32'), dtype('&lt;U32')) -&gt; dtype('&lt;U32')</pre><p> 起初，我认为它与数据集有关，但是，我没有更改我的数据集，并且在我第一次训练这个 model 时它已经运行。 </p></div></u32’)<></u32’),>

Question

I am using the Deep Pavlov framework to work with Bert Classifier simply because the language I need to predict staff is Russian. Basically, I am trying to solve a multi-class classification problem. According to the Deep Pavlov, we can easily change some configs on config file. I took this config file https://github.com/deepmipt/DeepPavlov/blob/master/deeppavlov/configs/classifiers/rusentiment_convers_bert.json and trained it, and it took me around 13 hours to finish and it turned out to be that my model is overfitting.

I made some changes, particularly these:

"weight_decay_rate": 0.001,
"learning_rate_drop_patience": 1,
"learning_rate_drop_div": 2.0,
"load_before_drop": True, 
"min_learning_rate": 1e-03,
"attention_probs_keep_prob": 0.5,
"hidden_keep_prob": 0.5,

also, I increased the batch size, it was 16 before now:

"batch_size": 32

and added some metrics:

"log_loss",
"matthews_correlation",

Also changed validation_patience to 1 and added tensorboard func

"validation_patience": 1,
"tensorboard_log_dir": "logs/",

and that is it. these are all the changes I made to my model, and when i tried to train my model, it is giving me following error:

UFuncTypeError                            Traceback (most recent call last)
        /usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py in _wrapfunc(obj, method, *args, **kwds)
             60     try:
        ---> 61         return bound(*args, **kwds)
             62     except TypeError:

15 frames
UFuncTypeError: ufunc 'clip' did not contain a loop with signature matching types (dtype('<U32'), dtype('<U32'), dtype('<U32')) -> dtype('<U32')

During handling of the above exception, another exception occurred:

UFuncTypeError                            Traceback (most recent call last)
<__array_function__ internals> in clip(*args, **kwargs)

/usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py in _clip_dep_invoke_with_casting(ufunc, out, casting, *args, **kwargs)
     83     # try to deal with broken casting rules
     84     try:
---> 85         return ufunc(*args, out=out, **kwargs)
     86     except _exceptions._UFuncOutputCastingError as e:
     87         # Numpy 1.17.0, 2019-02-24

UFuncTypeError: ufunc 'clip' did not contain a loop with signature matching types (dtype('<U32'), dtype('<U32'), dtype('<U32')) -> dtype('<U32')

At first, I thought it has something to do with a dataset, however, I did not change my dataset and it has run the first time I trained this model.

Answer 1

log_loss in DeepPavlov is just a wrapper over sklearn.metrics.log_loss: https://github.com/deepmipt/DeepPavlov/blob/master/deeppavlov/metrics/log_loss.py#L37

By default DeepPavlov uses chainer's out as y_pred in metrics computation and chainer's in_y as y_true .

To use log loss you can specify y_true as y or y_ids . And specify y_pred as y_pred_probas in log loss computation. This change will compute log loss for your case:

{
  "name": "log_loss",
  "inputs": [
    "y",
    "y_pred_probas"
  ]
}