creating ant file for eclipse project

Sometimes you need an ant build file for an existing Eclipse project, for instance when moving it to jenkins.. The easy way to do it is as follows:
on Eclipse click on File -> Export -> General -> Ant Buildfiles and specify the wanted fields.
It does not automatically have a war target, so have to add it.

selenium quickstart tips

By using Selenium (http://www.seleniumhq.org/), you can automate browsers and it is a good practice to use the tool to do some integration testing.. There are many tutorials about this on the web, the following worked for me:

• Selenium IDE Demo - Quick Beginner's Tutorial  mentions the Firefox plugin which records and plays tests. Selenium IDE is useful, you can easily save and replay tests as html files, but i want my tests written in java and want to run them as junit tests.
• The following three videos by Absoft Training  are great!!
• Selenium Tutorial - Starting with Selenium WebDriver – Part 1 mentions WebDriver interface and illustrates to write basic test in java. It also shows how to use it for cross browser testing.
• Selenium Tutorial - Starting with Selenium WebDriver – Part 2 explains WebElement in detail and shows how to locate an element on the web page.
• Using XPath to Locate Web Elements - in very detail shows how to locate an element by xpath.

Absoft Training presents more tutorials here: http://www.absofttrainings.com/selenium-training-free-videos/ 

My usage tips :

• Download Selenium standalone server and the java client.
• In Eclipse create a java project and add the downloaded jars (must be 2 jars) to classpath.
• Download Selenium IDE and install it as Firefox plugin.
• Run the server and deploy your application to test.
• Open Firefox, start Selenium IDE, press record button and enter your web application url.
• Test your application, one use case at a time.
• Close Selenium recording, save it and export test case as Java / Junit 4/ WebDriver.
• I did not use the immediately exported files. But, i got help from them. I think preparing a utility file which enables you to switch between browser types, between development and production environments is convenient. If you have a complicated web page, it is good practice to write find methods for web elements and reuse them.
• Then run tests as junit.

Some technical tips:

• WebDriver interface provides many methods including findElement(), getTitle(), close(), getCurrentUrl() etc. It is good to examine its javadoc.
• You can locate a web element by id, name or xpath.
• To create Firefox driver, simply :
• WebDriver driver = new FirefoxDriver();
• To create Chrome driver, download the third party executable
• System.setProperty("webdriver.chrome.driver", "path_to_executable");
• WebDriver driver = new ChromeDriver();
• To create IE driver, download the third party executable
• System.setProperty("webdriver.ie.driver", "path_to_executable");
• WebDriver driver = new InternetExplorerDriver();
• To make same arrangements:
• driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
• driver.manage().window().maximize(); // maximizes opening browser
• Selenium sees everything on a web page as web element.
• findElement retrieves first one that matches
• findElements retrieves all elements that match
• You can locate by
• Id
• Name
• LinkText (text of hyperlink)
• PartialLinkText
• Xpath (ultimate solution, when others don't work)
• ByXpath("//input[@placeholder='Email']") means locate an input field whose placeholder attribute value is Email.
• CssSelector
• TagName
• ClassName
• Locating by Xpath
• Xpath-Absolute: not recommended.
• //html/body/tag1[index]/..
• Xpath-Attributes: best when can't use id or name
• //input[@id='Pass' and @placeholder='Password']
• //input[contains(@id, 'user')] 
• //input[starts-with(@id, 'user')] 
• Selenium IDE and Firefox developer tools can help to get the xpath of the element.
• Xpath-Relative: best for table cells
• //table[@id='xtable']/tbody/tr[3]/td[2]
• You can make Selenium to wait for the presence of some element:
• final WebDriverWait wait = new WebDriverWait(driver, 60);
• wait.until(ExpectedConditions.presenceOfElementLocated(By.id("the_id")));

A test case template:

// create driver for the browser you want
WebDriver driver = new FirefoxDriver();

//open homepage
driver.get("the_url");

//find related field, enter some value
driver.findElement(By.[id|name|xpath]("the_value")).[clear()|click()|sendKeys("..")];

//find command button and click
driver.findElement(By.id("the_id")).click();

//assert results

mockito with testng suite

When writing tests by mocking your objects, you have to call a method in order to make sure your @Mock annotations really work. This method call is as follows:

 MockitoAnnotations.initMocks(this);  

In general, we use a BaseTest and call the method there and extend our tests from that BaseTest. Thus, the tests succeed when you run them separately. However, when you call them as suite, you may get the folowing error:

 testClass cannot be null. For info how to use @Mock annotations see examples in javadoc for MockitoAnnotations class  

I tried several approaches to fix this including using MockitoTestNGListener.class, none of which work. Then i find an article advising not to use inheritance in tests (http://www.petrikainulainen.net/programming/unit-testing/3-reasons-why-we-should-not-use-inheritance-in-our-tests/), removing inheritance and calling MockitoAnnotations.initMocks(this);  inside every test class worked :)

using testng on eclipse

This post provides a quick roadmap to start using testng on eclipse, since there is enourmous amount of documentation on web and it is hard to pick the most relevant ones..

• http://www.testng.org/doc/eclipse.html : quite well to start using the framework.
• https://khyatisehgal.wordpress.com/2014/08/24/java-lang-unsupportedclassversionerror-unsupported-major-minor-version-51-0-at/ : helps to solve the troubles encountered while following the first tutorial..
• https://www.youtube.com/watch?v=gFLpTjH9dlY : not too relevant, but it is a good seminar about testing on enterprise applications, so i wanted to share..

another tip: testng eclipse plugin version 6.9.5... (Update Site - http://beust.com/eclipse) supports JDK7 and 6.8.6 (Update Site - http://beust.com/eclipse-old/eclipse-6.8.6.20141201_2240) supports JDK6. 

Lucene tip

If you manually delete some files from the lucene index directory, than the next time you try to index you will get an error about the location being OUT-OF-SYNC. The error message may look like something like that:

 Entity com.mypackage.MyEntity Id 12345 Work Type org.hibernate.search.backend.UpdateLuceneWork  
 Entity com.mypackage.MyEntity Id 12346 Work Type org.hibernate.search.backend.UpdateLuceneWork  
 : org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/path/to/index/com.mypackage.MyEntity/write.lock  
 at org.apache.lucene.store.Lock.obtain(Lock.java:84) [lucene-core-3.5.0.jar:3.5.0 1204425 - simon - 2011-11-21 11:20:12]  
 at org.apache.lucene.index.IndexWriter.(IndexWriter.java:1108) [lucene-core-3.5.0.jar:3.5.0 1204425 - simon - 2011-11-21 11:20:12]  
 at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:127) [hibernate-search-engine-4.1.0.Final.jar:4.1.0.Final]  
 at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:102) [hibernate-search-engine-4.1.0.Final.jar:4.1.0.Final]  
 at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:119) [hibernate-search-engine-4.1.0.Final.jar:4.1.0.Final]  
 at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99) [hibernate-search-engine-4.1.0.Final.jar:4.1.0.Final]  
 at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67) [hibernate-search-engine-4.1.0.Final.jar:4.1.0.Final]  
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_03]  
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_03]  
 at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_03]  
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_03]  
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_03]  
 at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_03]  

My solution to this situation, is clearing the OUT-OF-SYNC folder content, restart the server and reindex.
More practical solutions may exist, but that worked for me.. By the way, just deleting the write.lock file does not work here..
Another tip about the lock mechanism:
(from http://www.gossamer-threads.com/lists/lucene/java-user/261725 :)

The existence of the write.lock file has nothing to do with actual locking. The lock file is just a placeholder file (0 bytes) on which the lock during indexing is applied (the actual locking is done on this file using a fnctl). When indexing finishes, the lock is removed from this file, but the file stays alive. This was different in Lucene 3, where the lock was also removed. But this caused problems because of non-atomic file operations. Lock factory only checks the "lock on the file", not "the existence of the file as a lock". 

Lucene on clustered environment

I have a task to refresh lucene indexes periodically. However, when the application is deployed on a clustered environment, each node tries to start indexing process and others display the following error:

 Entity com.me.MyEntity Id 1783 Work Type org.hibernate.search.backend.UpdateLuceneWork  
 : org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/path_to_index/com.me.MyEntity/write.lock  
 at org.apache.lucene.store.Lock.obtain(Lock.java:84) [lucene-core-3.5.0.jar:3.5.0 1204425 - simon - 2011-11-21 11:20:12]  
 at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1108) [lucene-core-3.5.0.jar:3.5.0 1204425 - simon - 2011-11-21 11:20:12]  
 at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:127) [hibernate-search-engine-4.1.0.Final.jar:4.1.0.Final]  
 at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:102) [hibernate-search-engine-4.1.0.Final.jar:4.1.0.Final]  
 at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:119) [hibernate-search-engine-4.1.0.Final.jar:4.1.0.Final]  
 at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99) [hibernate-search-engine-4.1.0.Final.jar:4.1.0.Final]  
 at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67) [hibernate-search-engine-4.1.0.Final.jar:4.1.0.Final]  
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_03]  
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_03]  
 at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_03]  
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_03]  
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_03]  
 at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_03]  

Those links may help you:
http://stackoverflow.com/questions/1228833/sharing-a-java-synchronized-block-across-a-cluster-or-using-a-global-lock 
http://stackoverflow.com/questions/17921898/using-lucene-in-a-clustered-environment-with-shared-nfs
http://forums.terracotta.org/forums/posts/list/6707.page
http://stackoverflow.com/questions/13486426/how-to-unlock-the-index-directory-in-lucene

But, i solved my case by adding a random timer to sleep :

 @Schedule(...)   
      public void myTask() {  
           try {  
                long sleepFor = (long)(Math.random() * 100000);  
                Thread.sleep(sleepFor); // hope nodes sleep different amount of time  
                log.infov("Sleep for this cluster : " + sleepFor);  
                Directory directory = FSDirectory.open(new File("path_to_index/com.me.MyEntity/"));  
             if (directory.fileExists("write.lock")) {  
               log.infov("ATTENTION!!! Already done by the other node!!");  
               return;  
             }  
             directory.close();  
 ...  

The random timer weakens the syncronization between nodes and the one who sleeps the least does the job! Then others know about this by checking the lock file..

Hope this helps..

lucene date issue

When indexing with Lucene (Hibernate Search), you may notice difference between dates in the database and indexed values. For example, in my case, Lucene dates were a day before the dates in the database. This is due to timezone differences. In the following annotation, org.hibernate.search.annotations.DateBridge assumes GMT time..

 @Column(name = "CREATED")  
 @Temporal(TemporalType.TIMESTAMP)  
 @DateBridge(resolution = Resolution.DAY)  
 private Date created;  

My solution was to write a custom field bridge named ESTDateBridge by copying the original DateBridge code and changing the default timezone from GMT to "America/New_York" (writing EST did not work..)
The working annotation is as follows:

 @Column(name = "CREATED")  
 @Temporal(TemporalType.TIMESTAMP)  
 @FieldBridge(impl = ESTDateBridge.class)  
 private Date created;  

Some references:
http://jamesjefferies.com/2009/04/22/hibernate-search-lucene-and-greenwich-mean-time/ 
https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Data_Grid/6.4/html/Infinispan_Query_Guide/sect-Bridges.html 
https://forum.hibernate.org/viewtopic.php?f=9&t=990266&view=next 

a simple tutorial about joins

Here is an excellent tutorial about basics of joins: https://www.youtube.com/watch?v=mhs4vwTSY-U

In fact, videos of Oresoft LWC seems very good for self training :  https://www.youtube.com/user/TheOresoft/playlists

Oracle performance tuning by Guy Harrison

I watched a great lecture on Oracle performance tuning by Guy Harrison and wanted to share it with my notes. The lecture can be watched here:
https://www.youtube.com/watch?v=Ah1xjCl6Axg   (part 1)
https://www.youtube.com/watch?v=f-C3nqaqND0 (part 2)
.. and my notes taken while watching:

• application --(sql)--> database server --(request blocks)--> server memory --(request blocks)--> IO subsytem

          Problems in one layer may be caused or cured in the higher layer.

          Therefore, tune db top-down 

• app code
• sql
• contention
• memory
• disk IO

• Make Oracle work smarter, not harder.. do not load it too much (donkey example in the lecture..)
• Nulls can not be indexed, so null equality queries work slower. Use some predefined value instead of nulls.
• Use concatenated index while searching multiple columns, reduces the IO requests drastically. but inserts get slower at the same time.
• Try to cache returned results, so do not go to database unless you need to.
• Use bind variables (otherwise -> parse overhead). use PreparedStatement in java.
• Minimize lock duration, use optimiztic locking
• Reduce network traffic by using array interface (setFetchSize) and stored procedure for complex sql s.
• help the optimizer (table statistics etc.)
• fix bad sql if optimizer makes suboptimal choice.
• Bad sql -> with highest CPU or IO consumption.
• V$SQL -> sys.dm_exec_requests
• order by, group by, join two tables without index -> performs a sort
• hash joins are more efficient than sort-merge
• not having enough memory to sort is bad..

shrinking tables to gain space in database (Oracle Sql Developer)

When you delete rows, the rows may not be physically deleted and you may not get the used space back.  The case is explained in the post: http://laurenthinoul.com/how-to-fix-ora-01654-unable-to-extend-index-in-tablespace/

You can run the following sql to compute statistics :

analyze table   compute statistics;


You can view the statistics as in the picture:

try this sql to shrink:

alter table  enable row movement;
alter table  shrink space;
alter table  disable row movement;

analyze table  compute statistics;
commit;


however, this may give the following error due to the reasons here: http://agstamy.blogspot.it/2009/02/ora-10631-when-trying-to-shrink-table.html

SQL Error: ORA-10631: SHRINK clause should not be specified for this object
 "SHRINK clause should not be specified for this object"
*Cause:    It is incorrect to issue shrink on the object
*Action:   Verify the object name and type and reissue the command

in this case, you may try by removing the indexes, shrinking the table and recreating the indexes.

btw, truncating the table gives you the space back!