Increase the number of results in elasticsearch

 curl -XGET '172.xx.xxx.xx:9200/document/post/_search?pretty=true' -d '{
"from" : 0, "size" : 30,
    "query" : { "query_string" : { "query": "assistance in voting" }},
    "highlight" : {
        "fields" : {
            "text" : {}
        }
    }
}'

http://www.elasticsearch.org/guide/reference/api/search/from-size.html

Start, stop Apache server on ubuntu

Task: Start Apache 2 Server

# /etc/init.d/apache2 start
or
$ sudo /etc/init.d/apache2 start

Task: Restart Apache 2 Server

# /etc/init.d/apache2 restart
or
$ sudo /etc/init.d/apache2 restart

Task: Stop Apache 2 Server

# /etc/init.d/apache2 stop
or
$ sudo /etc/init.d/apache2 stop

Highlighting search results with the query term in ElasticSearch

 curl -XGET 'http://localhost:9200/document/_search?pretty=true' -d '{
    "query" : { "query_string" : { "query": "assistance in voting" }},
    "highlight" : {
        "fields" : {
            "text" : {}
        }
    }
}'

MyFitnessPal vs Livestrong My Plate

Tried both apps today, My Plate might have a better user interface but overall I like MyFitnessPal, better database and ease of use.

Highlighting search results with the query term in ElasticSearch

Highlighting search results with the query term in ElasticSearch:

curl -XGET 'http://localhost:9200/document/_search?pretty=true' -d '{
    "query" : { "term" : { "title": "mobile" }},
    "highlight" : {
        "fields" : {
            "text" : {}
        }
    }
}'

Deleting documents from your index Apache Solr

How can I delete all documents from my index?

Use the "match all docs" query in a delete by query command: <delete><query>*:*</query></delete>

You must also commit after running the delete so, to empty the index, run the following two commands:

curl http://localhost:8983/solr/update --data '<delete><query>*:*</query></delete>' -H 'Content-type:text/xml; charset=utf-8'  
curl http://localhost:8983/solr/update --data '<commit/>' -H 'Content-type:text/xml; charset=utf-8'

Another strategy would be to add two bookmarks in your browser:

http://localhost:8983/solr/update?stream.body=<delete><query>*:*</query></delete>
http://localhost:8983/solr/update?stream.body=<commit/>

And use those as you're developing to clear out the index as necessary.

for loop in unix

for (( i = 1 ; i <= 10000; i++ )); do java -jar post.jar $i.xml; done

Apache SOLR

Apache Solr is an open source search server. It is based on the full text search engine called Apache Lucene. So basically Solr is an HTTP wrapper around an inverted index provided by Lucene. An inverted index could be seen as a list of words where each word-entry links to the documents it is contained in. That way getting all documents for the search query "dzone" is a simple 'get' operation.

One advantage of Solr in enterprise projects is that you don't need any Java code, although Java itself has to be installed. If you are unsure when to use Solr and when Lucene, these answers could help. If you need to build your Solr index from websites, you should take a look into the open source crawler called Apache Nutch before creating your own solution.

Unzipping a bz2 file without deleting the original file

To uncompress a bzip2 file, execute the following command

#bunzip2 filename.txt.bz2 (where filename.txt.bz2 is the name of the file you wish to uncompress)

The result of this operation is a file called filename.txt. By default, bunzip2 will delete the filename.txt.bz2 file.

Use -k to keep the original file intact, so the command becomes something like this:

#bunzip2 - k filename.txt.bz2 

Reference: http://www.debianadmin.com/create-and-extract-bz2-and-gz-files.html

Create and Extract .bz2 and .gz files

#apt-get install bzip2

Installing bzip2 in ubuntu

sudo apt-get install bzip2

Uncompressing a bzip2 File Using bunzip2 in Debian

To uncompress a bzip2 file, execute the following command

#bunzip2 filename.txt.bz2 (where filename.txt.bz2 is the name of the file you wish to uncompress)

Reference: http://www.debianadmin.com/create-and-extract-bz2-and-gz-files.html

Installing stemming porter library python

From the following URL download the egg file:

http://pypi.python.org/pypi/stemming/1.0.1

Once you have the file run the following command

sudo easy_install filename.egg

There you go :)

Changing the navbar color in Bootstrap

The best way currently to do the same would be to install LESS command line compiler using

$ npm install -g less jshint recess uglify-js

Once you have done this, then go to the less folder in the directory and then edit the file variables.less and you can change a lot of variables according to what you need including the color of the navigation bar

@navbarCollapseWidth:             979px;

@navbarHeight:                    40px;
@navbarBackgroundHighlight:       #ffffff;
@navbarBackground:                darken(@navbarBackgroundHighlight, 5%);
@navbarBorder:                    darken(@navbarBackground, 12%);

@navbarText:                      #777;
@navbarLinkColor:                 #777;
@navbarLinkColorHover:            @grayDark;
@navbarLinkColorActive:           @gray;
@navbarLinkBackgroundHover:       transparent;
@navbarLinkBackgroundActive:      darken(@navbarBackground, 5%);

Once you have done this, go to your bootstrap directory and run the command make.

No output on nohup.out when running a python program in the background

Python when you run it as a background process, it keeps the buffers to itself and does not output anything to your nohup.out, I was stuck at this problem for hours until jk helped me! The solution is to just use python -u

Yup that's it!

So instead of nohup python filename.py & use nohup python -u filename.py &

Few more important commands:

Getting a list of processes  
ps -ax | grep python

Force kill a process
kill -9 processid

Saving your LSI model in gensim

A description of errors and the code, and finally what worked. I think it was just a silly mistake. I was accessing the npy file whereas the normal pkl file access worked.

Error
LSI
Traceback (most recent call last):
  File "tutorial1.py", line 85, in <module>
    lsi.load('tmp/lsa_model.pkl.npy')

Code 

#lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)

print "\nLSI"

#lsi.save('tmp/lsa_model.pkl')

lsi = models.LsiModel.load('tmp/lsa_model.pkl.npy')

Error

Traceback (most recent call last):

  File "tutorial1.py", line 86, in <module>

    lsi = models.LsiModel.load('tmp/lsa_model.pkl.npy')

  File "/usr/local/lib/python2.7/dist-packages/gensim-0.8.6-py2.7.egg/gensim/models/lsimodel.py", line 533, in load

    result = utils.unpickle(fname)

  File "/usr/local/lib/python2.7/dist-packages/gensim-0.8.6-py2.7.egg/gensim/utils.py", line 492, in unpickle

    return cPickle.load(open(fname, 'rb'))

cPickle.UnpicklingError: invalid load key, '�'.

But finally this worked:

#lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)

print "\nLSI"

#lsi.save('tmp/lsa_model.pkl')

lsi = models.LsiModel.load('tmp/lsa_model.pkl')

So finally this is what worked:

Saving the LSA:

lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)

print "\nLSI"

lsi.save('tmp/lsa_model2.pkl')

#lsi = models.LsiModel.load('tmp/lsa_model.pkl')

Loading the LSA:

#lsi = models.LsiModel(corpus, id2word = dictionary, num_topics =6)

print "\nLSI"

#lsi.save('tmp/lsa_model2.pkl')

lsi = models.LsiModel.load('tmp/lsa_model2.pkl')

Add more stopwords to your gensim implementation

from gensim import corpora, models, similarities

# remove common words and tokenize
stoplist = set("a about above after again against all am an and any are aren't as at be because been before being below between both but by can't cannot could couldn't did didn't do does doesn't doing don't down during each few for from further had hadn't has hasn't have haven't having he he'd he'll he's her here here's hers herself him himself his how how's i i'd i'll i'm i've if in into is isn't it it's its itself let's me more most mustn't my myself no nor not of off on once only or other ought our ours  ourselves out over own same shan't she she'd she'll she's should shouldn't so some such than that that's the their theirs them themselves then there there's these they they'd they'll they're they've this those through to too under until up very was wasn't we we'd we'll we're we've were weren't what what's when when's where where's which while who who's whom why why's with won't would wouldn't you you'd you'll you're you've your yours yourself yourselves , .".split())

nohup Execute Commands After You Exit From a Shell Prompt

Most of the time you login into remote server via ssh. If you start a shell script or command and you exit (abort remote connection), the process / command will get killed. Sometime job or command takes a long time. If you are not sure when the job will finish, then it is better to leave job running in background. However, if you logout the system, the job will be stopped. What do you do?

nohup command

Answer is simple, use nohup utility which allows to run command./process or shell script that can continue running in the background after you log out from a shell:

nohup Syntax:

nohup command-name &

Where,

• command-name : is name of shell script or command name. You can pass argument to command or a shell script.
• & : nohup does not automatically put the command it runs in the background; you must do that explicitly, by ending the command line with an & symbol.

nohup command examples

1) Login to remote server
$ ssh user@remote.server.com
2) Execute script called pullftp.sh
# nohup pullftp.sh &

Reference: http://www.cyberciti.biz/tips/nohup-execute-commands-after-you-exit-from-a-shell-prompt.html

Spring MVC file upload NullPointerException

Took me hours to figure out and in the end I realized I just had to add this small piece of resolver in my dispatcher-servlet.


    <bean id="multipartResolver"
          class="org.springframework.web.multipart.commons.CommonsMultipartResolver">

        <!-- one of the properties available; the maximum file size in bytes -->
        <property name="maxUploadSize" value="100000"/>
    </bean>

PostgreSQL create table if not exists

Try running a query on the table. If it throws an exception then catch the exception and create a new table.

try {
    int a =  db.queryForInt("SELECT COUNT(*) FROM USERS;");
}
catch (Exception e) {
    System.out.print(e.toString());
    db.update("CREATE TABLE USERS (" +
                "id SERIAL," +
                "PRIMARY KEY(id)," +
                "name varchar(30) NOT NULL," +
                "email varchar(30) NOT NULL," +
                "username varchar(30) NOT NULL," +
                "password varchar(30) NOT NULL" +
                ");");
}
return db;

Example of Log4J

Log4J Example:

@Controller

public class VanityController {

    private static Logger logger = Logger.getLogger(VanityController.class.toString());

    @RequestMapping(value= "/user/{vanity}", method = RequestMethod.GET)

    public ModelAndView viewProfile(@PathVariable String vanity) {

        ModelAndView mv = new ModelAndView("vanity");

        logger.info(vanity + "vanity called!");

        mv.addObject("name", vanity);

        return mv;

    }

}

when called localhost:8080/user/todo

this is what was displayed in the output: 

INFO: todovanity called!

Adding Log4J dependency to Maven

<dependency>
      <groupId>log4j</groupId>
      <artifactId>log4j</artifactId>
      <version>1.2.14</version>
      <scope>runtime</scope>
   </dependency>

FATAL: password authentication failed for user postgres

If you face this error on trying to use psql, the following steps should fix it:

By default, postgres creates a user named 'postgres'. We log in as postgres, and give a password.

$ sudo -u postgres psql
\password
Enter password: ...
...

Now try again. The above steps work for a fresh install of postgres 9.1 and 8.4 on Ubuntu 12.04.

Installing postgresql in Ubuntu and configuring it with your SpringFramework application

Installation sudo apt-get install postgresql

Installing the GUI interface sudo apt-get install pgadmin3

Starting psql on command line: sudo -u postgres psql postgres

                                                                  username       dbname

postgres is the user/admin which is there by default.

Also you can this on the command line to start psql: psql -u postgres -h localhost

Creating a new db  sudo -u postgres createdb mydb1 or you can use the GUI to add a new db. 

Starting psql for your db sudo -u postgres psql mydb1

Time to configure postgresql to your Spring application, we assume that you are using Maven as the build tool.

Add the following dependency in your pom.xml

<dependency>

<groupId>postgresql</groupId>

<artifactId>postgresql</artifactId>

<version>9.1-901.jdbc4</version>

</dependency>

But before this make sure you check the version of your postgres by writing on the command line

psql --version, the above code is valid if your version is 9.1.4 else go here http://mvnrepository.com/artifact/postgresql/postgresql/ and add the dependency to your pom.xml accordingly. 

            

Checking that your code is working by creating a connection to the Database. If the following code does not give any error then you have a reason to celebrate. Make sure to add the try and catch blocks wherever necessary.

  Class.forName("org.postgresql.Driver");

  Connection connection = DriverManager.getConnection(

                    "jdbc:postgresql://127.0.0.1:5432/mydb1", "postgres",

                    "crossword");

Now we need to add a Configuration file in our SpringFramework that will have the configuration for the datasource to connect to postgres. Your configuration will have the following datasource:

        BasicDataSource dataSource = new BasicDataSource();

        dataSource.setUrl("jdbc:postgresql://127.0.0.1:5432/mydb1");

        dataSource.setDriverClassName("org.postgresql.Driver");

        dataSource.setUsername("postgres");

        dataSource.setPassword("crossword");

        SimpleJdbcTemplate db = new SimpleJdbcTemplate(dataSource);


After making these changes this is the commit:
https://github.com/gauravmunjal/HelloWorldSpring/commit/51041b61b7230816175e53fe47b78132fd762f39

References 

http://www.postgresql.org/docs/9.1/interactive/tutorial-table.html

http://www.mkyong.com/jdbc/how-do-connect-to-postgresql-with-jdbc-driver-java/

Using Springframework in Idea Intellij with Maven Build Tool

• Create a new Project from Scratch
• Select the Maven Module
• Maven allows a project to build using its project object model (POM) and a set of plugins that are shared by all projects using Maven, providing a uniform build system. Once you familiarize yourself with how one Maven project builds you automatically know how all Maven projects build saving you immense amounts of time when trying to navigate many projects.
• Add the following dependencies to your pom.xml
• <dependencies>
          <dependency>
              <groupId>commons-dbcp</groupId>
              <artifactId>commons-dbcp</artifactId>
              <version>1.4</version>
          </dependency>
          <dependency>
              <groupId>com.google.collections</groupId>
              <artifactId>google-collections</artifactId>
              <version>1.0</version>
          </dependency>
          <dependency>
              <groupId>org.springframework</groupId>
              <artifactId>spring-jdbc</artifactId>
              <version>3.0.5.RELEASE</version>
          </dependency>
          <dependency>
              <groupId>org.springframework</groupId>
              <artifactId>spring-webmvc</artifactId>
              <version>3.0.5.RELEASE</version>
          </dependency>
          <dependency>
              <groupId>cglib</groupId>
              <artifactId>cglib</artifactId>
              <version>2.2.2</version>
          </dependency>
          <dependency>
              <groupId>org.codehaus.jackson</groupId>
              <artifactId>jackson-mapper-asl</artifactId>
              <version>1.5.3</version>
          </dependency>
          <dependency>
              <groupId>org.hsqldb</groupId>
              <artifactId>hsqldb</artifactId>
              <version>2.0.0</version>
          </dependency>
          <dependency>
              <groupId>javax.servlet</groupId>
              <artifactId>jstl</artifactId>
              <version>1.2</version>
          </dependency>
          <dependency>
              <groupId>javax.servlet</groupId>
              <artifactId>servlet-api</artifactId>
              <version>2.5</version>
          </dependency>
      </dependencies>
• Build configuration in pom.xml
• <build>

        <finalName>YourApplicationName</finalName>

        <plugins>

            <plugin>

                <artifactId>maven-compiler-plugin</artifactId>

                <configuration>

                    <source>1.6</source>

                    <target>1.6</target>

                    <encoding>UTF-8</encoding>

                </configuration>

            </plugin>

            <plugin>

                <groupId>org.mortbay.jetty</groupId>

                <artifactId>jetty-maven-plugin</artifactId>

                <version>7.4.2.v20110526</version>

                <configuration>

                    <scanIntervalSeconds>1</scanIntervalSeconds>

                </configuration>

            </plugin>

        </plugins>

    </build>

• In the java package create a new package with your desired name and then add three more packages namely controllers, model and services
• In the main directory create a directory webapp and in it create two directories static and WEB-INF
• In the WEB-INF directory add a new directory 'jsp' and in it create a jsp file 
• Also the WEB-INF Directory should have two more files dispatcher-servlet.xml and web.xml 
• dispatcher-servlet.xml

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"

       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

       xmlns:context="http://www.springframework.org/schema/context"

       xmlns:mvc="http://www.springframework.org/schema/mvc"

       xsi:schemaLocation="http://www.springframework.org/schema/beans

        http://www.springframework.org/schema/beans/spring-beans-3.0.xsd

        http://www.springframework.org/schema/context

        http://www.springframework.org/schema/context/spring-context-3.0.xsd

        http://www.springframework.org/schema/mvc

        http://www.springframework.org/schema/mvc/spring-mvc.xsd">

    <mvc:annotation-driven/>

    <context:component-scan base-package="SampleApp"/>

    <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">

        <property name="viewClass" value="org.springframework.web.servlet.view.JstlView"/>

        <property name="prefix" value="/WEB-INF/jsp/"/>

        <property name="suffix" value=".jsp"/>

    </bean>

</beans>

• web.xml

<?xml version="1.0" encoding="UTF-8"?>

<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

         xmlns="http://java.sun.com/xml/ns/javaee"

         xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"

         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"

         id="WebApp_ID"

         version="2.5">

    <display-name>Archetype Created Web Application</display-name>

    <servlet-mapping>

        <servlet-name>default</servlet-name>

        <url-pattern>/static/*</url-pattern>

    </servlet-mapping>

    <servlet>

        <servlet-name>dispatcher</servlet-name>

        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>

        <load-on-startup>1</load-on-startup>

    </servlet>

    <servlet-mapping>

        <servlet-name>dispatcher</servlet-name>

        <url-pattern>/</url-pattern>

    </servlet-mapping>

</web-app>

After doing the same, just compile your code using Maven and run the Jetty server whose dependency was added in the pom.xml file. Here is the link to the public repository for the same: https://github.com/gauravmunjal/HelloWorldSpring

Exception handling in Java

An exception is an event, which occurs during the execution of a program, that disrupts the normal flow of the program's instructions. When an error occurs within a method, the method creates an object and hands it off to the runtime system. The object, called an exception object, contains information about the error, including its type and the state of the program when the error occurred. Creating an exception object and handing it to the runtime system is called throwing an exception. 

An exception is a problem that arises during the execution of a program. An exception can occur for many different reasons, including the following:

1. A user has entered invalid data.
2. A file that needs to be opened cannot be found.
3. A network connection has been lost in the middle of communications, or the JVM has run out of memory.

Some of these exceptions are caused by user error, others by programmer error, and others by physical resources that have failed in some manner.

To understand how exception handling works in Java, you need to understand the three categories of exceptions:

1. Checked exceptions: A checked exception is an exception that is typically a user error or a problem that cannot be foreseen by the programmer. For example, if a file is to be opened, but the file cannot be found, an exception occurs. These exceptions cannot simply be ignored at the time of compilation.
2. Runtime exceptions: A runtime exception is an exception that occurs that probably could have been avoided by the programmer. As opposed to checked exceptions, runtime exceptions are ignored at the time of compliation.
3. Errors: These are not exceptions at all, but problems that arise beyond the control of the user or the programmer. Errors are typically ignored in your code because you can rarely do anything about an error. For example, if a stack overflow occurs, an error will arise. They are also ignored at the time of compilation.

Binary Search Tree

In computer science, a binary search tree (BST), which may sometimes also be called an ordered or sorted binary tree, is a node-based binary tree data structure which has the following properties:

• The left subtree of a node contains only nodes with keys less than the node's key.
• The right subtree of a node contains only nodes with keys greater than the node's key.
• Both the left and right subtrees must also be binary search trees.

A basic and a quick video, might be messed up too in some places, explaining the basics of a Binary Search Tree.

Java: Introduction to Strings

A string is an ordered sequence of symbols. Sting class is included in java.lang package. So each class in Java, can use String class without importing any package. The String class represents character strings. All string literals in Java programs, such as "abc", are implemented as instances of this class. 

Documentation:

http://download.oracle.com/javase/1.4.2/docs/api/java/lang/String.html

Comparing Strings:

http://www.cs.geneseo.edu/~baldwin/reference/java-strings.html

Minimax Algorithm

Consider a game of two players, call one of the players as MAX and the other player as MIN. And in the game MAX makes the first move and then they take turns alternatively till the end of the game.

The various components of the Game:

1. Initial State: This includes the board position and the player who will move first.
2. Successor Function: For a node it returns the list of it's successor function and the state they will result into.
3. Terminal States: The states where the game has ended.
4. Utility Function: This gives a numerical value to all the terminal states. High values are considered good for MAX and bad for MIN and vice-versa.

The following video gives a complete explanation of the Minimax Algorithm.

Following is the Minimax Algorithm:

Source: Artificial Intelligence: A Modern Approach

Installing Google App Engine Plugin on Eclipse Indigo

While installing Google App engine plugin for Eclipse Indigo, the installation process seem to stop as soon the the installation process reaches around 40% completion. I tried installing the same thrice and the installation ceases at the same point everytime. As a solution I switched to Eclipse Helios.

Starting a simple HTTPServer in python

Python comes with its own web server included in the http.server library module. 

To start a web server in python:

from http.server import HTTPServer, CGIHTTPRequestHandler

port = 8080

httpd = HTTPServer((' ',port), CGIHTTPRequestHandler)

httpd.server_forever()

After starting the server when you run localhost:8080 on your browser it shows you the current directory from where the server was started. That can be changed by importing the os module.

Accessing the server on the network:

After starting the server, create a wireless network. When another computer connects to the network, the server from the first computer can be accessed using the IP address and port number. To find the IP address. 

Run the following command to find the ip address:

ifconfig

After finding the IP address of the machine where the server is hosted, using the IP address and the port number the server can be accessed on the other machine.

[IP Address]:8080