Pages

Friday, October 5, 2012

Saving your LSI model in gensim


A description of errors and the code, and finally what worked. I think it was just a silly mistake. I was accessing the npy file whereas the normal pkl file access worked.

Error
LSI
Traceback (most recent call last):
  File "tutorial1.py", line 85, in <module>
    lsi.load('tmp/lsa_model.pkl.npy')

Code 
#lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)
print "\nLSI"
#lsi.save('tmp/lsa_model.pkl')
lsi = models.LsiModel.load('tmp/lsa_model.pkl.npy')

Error
Traceback (most recent call last):
  File "tutorial1.py", line 86, in <module>
    lsi = models.LsiModel.load('tmp/lsa_model.pkl.npy')
  File "/usr/local/lib/python2.7/dist-packages/gensim-0.8.6-py2.7.egg/gensim/models/lsimodel.py", line 533, in load
    result = utils.unpickle(fname)
  File "/usr/local/lib/python2.7/dist-packages/gensim-0.8.6-py2.7.egg/gensim/utils.py", line 492, in unpickle
    return cPickle.load(open(fname, 'rb'))
cPickle.UnpicklingError: invalid load key, '�'.

But finally this worked:

#lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)
print "\nLSI"
#lsi.save('tmp/lsa_model.pkl')
lsi = models.LsiModel.load('tmp/lsa_model.pkl')

So finally this is what worked:

Saving the LSA:

lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)
print "\nLSI"
lsi.save('tmp/lsa_model2.pkl')
#lsi = models.LsiModel.load('tmp/lsa_model.pkl')

Loading the LSA:

#lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)
print "\nLSI"
#lsi.save('tmp/lsa_model2.pkl')
lsi = models.LsiModel.load('tmp/lsa_model2.pkl')

No comments:

Post a Comment