pycocoevalcap代码的使用和问题解决
github链接
https://github.com/232525/PureT
总结
按照下面改了好多， 总是出错， 最后我重启了电脑，并使用通过pip安装的pycocoevalcap和java1.8能够完整运行，nice，真是无语
pycocoevalcap 1.2 pypi_0 pypi
pycocotools 2.0.6 pypi_0 pypi
pycocotools-windows 2.0.0.2 pypi_0 pypi
java的下载 https://blog.csdn.net/JunLeon/article/details/122623465
下载官网：Java Downloads | Oracle
下载版本：jdk-8u321-windows-x64.exe
百度网盘：链接：https://pan.baidu.com/s/1F43G56vjd5JtZXxbkQJlIA 提取码：8888
windows安装wegt
windows10下运行.sh文件报错：wget: command not found
https://blog.csdn.net/buluxianfeng/article/details/116049447
按照readme.md处理
Demo
https://blog.csdn.net/weixin_41848012/article/details/121254472
https://zhuanlan.zhihu.com/p/613433647
# -*- coding=utf-8 -*-
# author: w61
# Test for several ways to compute the score of the generated words.
from pycocoevalcap.bleu.bleu import Bleu
from pycocoevalcap.meteor.meteor import Meteor
from pycocoevalcap.rouge.rouge import Rouge
from pycocoevalcap.cider.cider import Cider
from pycocoevalcap.spice.spice import Spice
from pycocoevalcap.wmd.wmd import WMD
class Scorer():
def __init__(self,ref,gt):
self.ref = ref
self.gt = gt
print('setting up scorers...')
self.scorers = [
(Bleu(4), ["Bleu_1", "Bleu_2", "Bleu_3", "Bleu_4"]),
(Meteor(),"METEOR"),
(Rouge(), "ROUGE_L"),
(Cider(), "CIDEr"),
(Spice(), "SPICE"),
(WMD(), "WMD"),
]
def compute_scores(self):
total_scores = {}
for scorer, method in self.scorers:
print('computing %s score...'%(scorer.method()))
score, scores = scorer.compute_score(self.gt, self.ref)
if type(method) == list:
for sc, scs, m in zip(score, scores, method):
print("%s: %0.3f"%(m, sc))
total_scores["Bleu"] = score
else:
print("%s: %0.3f"%(method, score))
total_scores[method] = score
print('*****DONE*****')
for key,value in total_scores.items():
print('{}:{}'.format(key,value))
if __name__ == '__main__':
ref = {
'1':['go down the stairs and stop at the bottom .'],
'2':['this is a cat.']
}
gt = {
'1':['Walk down the steps and stop at the bottom. ', 'Go down the stairs and wait at the bottom.','Once at the top of the stairway, walk down the spiral staircase all the way to the bottom floor. Once you have left the stairs you are in a foyer and that indicates you are at your destination.'],
'2':['It is a cat.','There is a cat over there.','cat over there.']
}
# 注意，这里如果只有一个sample，cider算出来会是0，详情请看评论区。
scorer = Scorer(ref,gt)
scorer.compute_scores()
自己通过pip install的可能没有WMD,可以将其注释掉
问题
cocoEvalCapDemo.ipynb
1、发生异常: FileNotFoundError (note: full exception trace is shown but execution is paused at: _run_module_as_main)
[WinError 2] 系统找不到指定的文件。
执行my_image_captioning_CNN\image_captioning_to_cnn\coco_caption\cocoEvalCapDemo.ipynb的时候出现的错误。
错误代码如下；
# create cocoEval object by taking coco and cocoRes
cocoEval = COCOEvalCap(coco, cocoRes)
# evaluate on a subset of images by setting
# cocoEval.params['image_id'] = cocoRes.getImgIds()
# please remove this line when evaluating the full validation set
cocoEval.params['image_id'] = cocoRes.getImgIds()
# evaluate results
# SPICE will take a few minutes the first time, but speeds up due to caching
cocoEval.evaluate() -------------------------------------------------------------------》错误
|
|
|
def evaluate(self):
imgIds = self.params['image_id']
# imgIds = self.coco.getImgIds()
gts = {}
res = {}
for imgId in imgIds:
gts[imgId] = self.coco.imgToAnns[imgId]
res[imgId] = self.cocoRes.imgToAnns[imgId]
# =================================================
# Set up scorers
# =================================================
print('tokenization...')
tokenizer = PTBTokenizer()
gts = tokenizer.tokenize(gts) ------------------------------------------------》错误
res = tokenizer.tokenize(res)
|
|
|
cmd.append(os.path.basename(tmp_file.name))
p_tokenizer = subprocess.Popen(cmd, cwd=path_to_jar_dirname, stdout=subprocess.PIPE)
------------------------------------------------------------------------------》 错误
错误信息如下：
发生异常: FileNotFoundError (note: full exception trace is shown but execution is paused at: _run_module_as_main)
[WinError 2] 系统找不到指定的文件。
参考解决的方法的网址：https://stackoverflow.com/questions/73193119/python-filenotfounderror-winerror-2-the-system-cannot-find-the-file-specifie
解决方法：
1、安装Java JDK1.8
2、将subprocess.py中找到如下代码
def __init__(self, args, bufsize=-1, executable=None,
stdin=None, stdout=None, stderr=None,
preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
shell=False, cwd=None, env=None, universal_newlines=False,
startupinfo=None, creationflags=0,
restore_signals=True, start_new_session=False,
pass_fds=(), *, encoding=None, errors=None):
将shell=False更改为：shell=True
完美解决
2、发生异常: OSError (note: full exception trace is shown but execution is paused at: _run_module_as_main)
[Errno 22] Invalid argument
计算computing METEOR score…的时候爆出的错误‘
https://github.com/wangleihitcs/CaptionMetrics/issues/1
对于以上等问题的解决
安装java后重启电脑
https://zhuanlan.zhihu.com/p/159204903
https://blog.csdn.net/Bit_Coders/article/details/120840271
METEOR score
https://github.com/wangleihitcs/CaptionMetrics/issues/1
https://github.com/tylin/coco-caption/issues/8
修改后的pycocoevalcap(我个人没有验证)
https://github.com/salaniz/pycocoevalcap
Spice
Threads( StanfordCoreNLP ) [0.667 seconds]
Error: Could not score batched file input:
org.fusesource.lmdbjni.LMDBException: 磁盘空间不足。
at org.fusesource.lmdbjni.Util.checkErrorCode(Util.java:44)
at org.fusesource.lmdbjni.Env.open(Env.java:192)
at org.fusesource.lmdbjni.Env.open(Env.java:72)
at org.fusesource.lmdbjni.Env.open(Env.java:65)
at edu.anu.spice.LmdbTupleDB.putTransaction(LmdbTupleDB.java:69)
at edu.anu.spice.SpiceParser.loadTuplesFromDB(SpiceParser.java:216)
at edu.anu.spice.SpiceParser.loadTuples(SpiceParser.java:245)
at edu.anu.spice.SpiceParser.parseCaptions(SpiceParser.java:251)
at edu.anu.spice.SpiceScorer.scoreBatch(SpiceScorer.java:109)
at edu.anu.spice.SpiceScorer.main(SpiceScorer.java:60)
Traceback (most recent call last):
File "h:/thrid_paper/github_code/my_image_captioning_CNN/image_captioning_to_cnn/eval_demo.py", line 53, in <module>
scorer.compute_scores()
File "h:/thrid_paper/github_code/my_image_captioning_CNN/image_captioning_to_cnn/eval_demo.py", line 29, in compute_scores
score, scores = scorer.compute_score(self.gt, self.ref)
File "h:\thrid_paper\github_code\my_image_captioning_CNN\image_captioning_to_cnn\coco_caption\pycocoevalcap\spice\spice.py", line 75, in compute_score
subprocess.check_call(spice_cmd,
File "H:\thrid_paper\github_code\image_caption\image_captioning_env\lib\subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['java', '-jar', '-Xmx8G', 'spice-1.0.jar', 'h:\\thrid_paper\\github_code\\my_image_captioning_CNN\\image_captioning_to_cnn\\coco_caption\\pycocoevalcap\\spice\\tmp\\tmpm4frve7c', '-cache', 'h:\\thrid_paper\\github_code\\my_image_captioning_CNN\\image_captioning_to_cnn\\coco_caption\\pycocoevalcap\\spice\\cache', '-out', 'h:\\thrid_paper\\github_code\\my_image_captioning_CNN\\image_captioning_to_cnn\\coco_caption\\pycocoevalcap\\spice\\tmp\\tmph6xire8e', '-subset', '-silent']' returned non-zero exit status 1.
==解决办法= =
在spice.py中更改一下代码
if not os.path.exists(cache_dir):
os.makedirs(cache_dir)
spice_cmd = ['java', '-jar', '-Xmx8G', SPICE_JAR, in_file.name,
# '-cache', cache_dir,
'-out', out_file.name,
'-subset',
'-silent'
]
注释掉 ： # ‘-cache’, cache_dir,、
WMD
from ot import emd2
ModuleNotFoundError: No module named ‘ot’
pip install POT