Pages

Friday, October 5, 2012

No output on nohup.out when running a python program in the background

Python when you run it as a background process, it keeps the buffers to itself and does not output anything to your nohup.out, I was stuck at this problem for hours until jk helped me! The solution is to just use python -u

Yup that's it!

So instead of nohup python filename.py & use nohup python -u filename.py &

Few more important commands:

Getting a list of processes  
ps -ax | grep python

Force kill a process
kill -9 processid

Saving your LSI model in gensim


A description of errors and the code, and finally what worked. I think it was just a silly mistake. I was accessing the npy file whereas the normal pkl file access worked.

Error
LSI
Traceback (most recent call last):
  File "tutorial1.py", line 85, in <module>
    lsi.load('tmp/lsa_model.pkl.npy')

Code 
#lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)
print "\nLSI"
#lsi.save('tmp/lsa_model.pkl')
lsi = models.LsiModel.load('tmp/lsa_model.pkl.npy')

Error
Traceback (most recent call last):
  File "tutorial1.py", line 86, in <module>
    lsi = models.LsiModel.load('tmp/lsa_model.pkl.npy')
  File "/usr/local/lib/python2.7/dist-packages/gensim-0.8.6-py2.7.egg/gensim/models/lsimodel.py", line 533, in load
    result = utils.unpickle(fname)
  File "/usr/local/lib/python2.7/dist-packages/gensim-0.8.6-py2.7.egg/gensim/utils.py", line 492, in unpickle
    return cPickle.load(open(fname, 'rb'))
cPickle.UnpicklingError: invalid load key, '�'.

But finally this worked:

#lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)
print "\nLSI"
#lsi.save('tmp/lsa_model.pkl')
lsi = models.LsiModel.load('tmp/lsa_model.pkl')

So finally this is what worked:

Saving the LSA:

lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)
print "\nLSI"
lsi.save('tmp/lsa_model2.pkl')
#lsi = models.LsiModel.load('tmp/lsa_model.pkl')

Loading the LSA:

#lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)
print "\nLSI"
#lsi.save('tmp/lsa_model2.pkl')
lsi = models.LsiModel.load('tmp/lsa_model2.pkl')

Thursday, October 4, 2012

Add more stopwords to your gensim implementation


from gensim import corpora, models, similarities

# remove common words and tokenize
stoplist = set("a about above after again against all am an and any are aren't as at be because been before being below between both but by can't cannot could couldn't did didn't do does doesn't doing don't down during each few for from further had hadn't has hasn't have haven't having he he'd he'll he's her here here's hers herself him himself his how how's i i'd i'll i'm i've if in into is isn't it it's its itself let's me more most mustn't my myself no nor not of off on once only or other ought our ours  ourselves out over own same shan't she she'd she'll she's should shouldn't so some such than that that's the their theirs them themselves then there there's these they they'd they'll they're they've this those through to too under until up very was wasn't we we'd we'll we're we've were weren't what what's when when's where where's which while who who's whom why why's with won't would wouldn't you you'd you'll you're you've your yours yourself yourselves , .".split())

Wednesday, October 3, 2012

nohup Execute Commands After You Exit From a Shell Prompt


Most of the time you login into remote server via ssh. If you start a shell script or command and you exit (abort remote connection), the process / command will get killed. Sometime job or command takes a long time. If you are not sure when the job will finish, then it is better to leave job running in background. However, if you logout the system, the job will be stopped. What do you do?

nohup command

Answer is simple, use nohup utility which allows to run command./process or shell script that can continue running in the background after you log out from a shell:

nohup Syntax:

nohup command-name &
Where,
  • command-name : is name of shell script or command name. You can pass argument to command or a shell script.
  • & : nohup does not automatically put the command it runs in the background; you must do that explicitly, by ending the command line with an & symbol.

nohup command examples

1) Login to remote server
$ ssh user@remote.server.com
2) Execute script called pullftp.sh
# nohup pullftp.sh &

Monday, August 13, 2012

Spring MVC file upload NullPointerException

Took me hours to figure out and in the end I realized I just had to add this small piece of resolver in my dispatcher-servlet.


    <bean id="multipartResolver"
          class="org.springframework.web.multipart.commons.CommonsMultipartResolver">

        <!-- one of the properties available; the maximum file size in bytes -->
        <property name="maxUploadSize" value="100000"/>
    </bean>