Fixing jExcel’s performance issue while running in a web application

Hopefully this post will help people who are experiencing performance problems with jExcel inside of a web application environment like Apache Tomcat, Websphere, and the like.  The short version of the post is this: if you’re experiencing a lot of slow-down while jExcel is being called set this switch in your web-application environment: jxl.nogc=true.  If you’re using Tomcat you’d modify the CATALINA_OPTS setting to be something like this:

CATALINA_OPTS=”…. -Djxl.nogc=true ….”

Now, the full explanation:

I use jExcel to create Excel documents on-the-fly.  It works really well, is simple to use, and is extremely fast; however, we were running into issues after creating the first Excel document.  The time spent creating the Excel spreadsheets to be written increased more than seven times after the first spreadsheet document was created.  Of course, this unacceptable, so I was tasked with finding the solution.

I loaded up the jExcel source code and began tracing through it figuring out how Excel documents are created within jExcel.  While I was searching, I was sent the following tidbit of information by one of my coworkers who was looking at the high level and stumbled upon it through this website (I would have eventually found this bit comment, but I’m glad he did since it saved me a lot of time).  The important bit of information is this in the WorkbookSettings class:

/**
* Flag to indicate whether the system hint garbage collection
* is enabled or not.
* As a rule of thumb, it is desirable to enable garbage collection
* when reading large spreadsheets from a batch process or from the
* command line, but better to deactivate the feature when reading
* large spreadsheets within a WAS, as the calls to System.gc() not
* only garbage collect the junk in JExcelApi, but also in the
* webservers JVM and can cause significant slowdown
* GC deactivated using -Djxl.nogc=true on the JVM command line
* Activated by default or by using -Djxl.nogc=false on the JVM command line
*/

The switch jxl.nogc was set to deactivate the manual call to Java’s garbage collection and the time spent setting up the Excel workbooks dropped by a factor of six from when we first tested the application.  Without the nogc activated jExcel performs a full garbage collection after certain operations are done within the jExcel code.  Which, as you can guess, full garbage collection while handling a request destroys any sort of quick response time that you might have been hoping for.

Cheers,

Ujjwal Soni

Netbeans plugin for AVR microcontrollers

Hi,

A new set of plugins for the NetBeans IDE that allows an embedded engineer to compile and upload their code to a microcontroller by utilizing AVRdude + extra features described in this blog post: http://netbeans.dzone.com/nb-c-avrdude-1 

Cheers,

Ujjwal Soni

Arduino + LCD

In this post I talk briefly about how to use the JHD 162A LCD module which connects to Arduino and powered by Seller LiquidCrystal.

The diagram is used to connect the following.

After configuring the electrical circuit, the programming side will do the same and tell the Arduino pin will be used so that, after that a message is displayed on the screen as show the following code example showing the famous Hello World :

#include

LiquidCrystal lcd(12,11,5,4,3,2);

void setup(){
  lcd.begin(16,2);
  pinMode(13,OUTPUT);
  digitalWrite(13, HIGH);
  Serial.begin(9600);
}

void loop(){
  LCD.write("Hola Mundo");
}

Cheers,

Ujjwal Soni

Windows Master Control Panel Shortcut aka ‘GOD MODE’

This is a lesser-known flaw in Windows 7 which has been touted as God Mode in the internet circles.

This is a short-cut to access various control settings in Windows Vista and later OS. By creation of a folder with a certain extension, users can access all of the OS’s control panels from within a single shortcut. The hack was published by 3rd parties outside of Microsoft documentation in 2007 and made popular by people claiming to have developed/discovered a ‘GOD MODE’ in windows.  Microsoft claims to have built it for developers only and advises against its use.

Process

Create  a new folder and rename it as: GodMode.{ED7BA470-8E54-465E-825C-99712043E01C}

Variations

The text ”GodMode” has nothing to do with making the trick work.  You can rename the folder “NickyGoesNutsAndBolts.{ED7BA470-8E54-465E-825C-99712043E01C}” and now you’ve discovered the magical “NickyGoesNutsAndBolts” feature hidden in Windows.

Why a Flaw?

GUID-based folder names can be exploited by computer worms—malicious software that can “reproduce”—hidden in a folder posing as a system folder such as the Recycle Bin, by employing its system application GUID shortcut. For example, a malicious file located at E:\Recycler\bin.{645ff040-5081-101b-9f08-00aa002f954e}\VIRUS.exe cannot be viewed in Explorer (opening the folder in order to view/delete would redirect to the Recycle Bin) and the folder cannot be deleted while the worm is running. Such a shortcut can be bypassed with the use of an active antivirus, or by booting from another operating system.

Validity and Future

This feature works on Windows Vista(x86/x64), Windows 7(x86/x64), Windows 8(x86/x64). Use of this shortcut in development of applications has been negatively advised by Microsoft as it is to be revoked in future editions of the OS.

Source: Wikipedia, MSDN

Windows authentication and fake/local domains defined in your host file

Hi,

I like to create my own local domains when creating a website on my computer as it’s both easier to work with and to remember than using the default site with a lot of virtual directories/applications. However if you want to use windows authentication on a fake domain like this you will run into problems. If you create your own fake domain on a computer using the host file (for instance www.test.local) and want to enable windows authentication on the site this won’t work out of the box. Here are some pointers on how to get this to work properly.

My step by step on how to add and use such domains are:

1. Edit C:\windows\system32\drivers\etc\hosts

You might have to change the security settings on this file to be able to edit this file.

Add in the hosts file.

Remember to never use a domain that could exist (like www.test.com). Always use a top domain that doesn’t exist.

2. Create your website in Internet Information Server Manager

Fill in the hostname field with the domain entered in your hosts file like below:

3. Configure your site to use Windows Authentication.

This is done in the Internet Information Server Manager by clicking on your site and then the Authentication icon. Disable all options but Windows Authentication and if needed ASP.NET Impersonation.

Select Windows Authentication in this list and click Providers on the right hand side options. Make sure you use the correct provider depending if you use NTLM or Kerberos as your verification method. If you’re not sure you can always try each option. Keep just one option provider enabled at a time (at least at first).

Do a iisreset from command prompt after you change the providers to make sure no setting is cached.

4. Add local domain support in registry.
Even if you’ve done everything above which are the typical steps to enable windows authentication on a normal web site using a proper domain it might not/won’t work using a host name specified domain.

To enable this feature you will have to enter some values into your registry. Start regedit and go to the HKLM\System\CurrentControlSet\Control\Lsa\MSV1_0 key.

Add a multi string value at this key named BackConnectionHostNames.

Enter the domain(s) you’ve added to your hostfile that you want windows authentication support for line by line like illustrated below.

Cheers,

Ujjwal Soni

Compress Images on upload

Hi,


Recently i wanted to compress images when uploading using Postgresql and hibernate. I implemented below code :

Image compressor utility code below ::

package com.utils;

import java.awt.Graphics2D; 
import java.awt.Image;
import java.awt.image.BufferedImage;
import java.awt.image.renderable.ParameterBlock;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.util.Iterator;

import javax.imageio.IIOImage;
import javax.imageio.ImageIO;
import javax.imageio.ImageWriteParam;
import javax.imageio.ImageWriter;
import javax.imageio.stream.MemoryCacheImageOutputStream;
import javax.media.jai.JAI;
import javax.media.jai.RenderedOp;

import com.sun.media.jai.codec.SeekableStream;

/**
 * 
 * @author Ujjwal Soni
 * 
 */

public class ImageCompressor {

    public byte[] compressFile(byte[] content, String fileName)
            throws FileNotFoundException {

        BufferedImage input = null;
        InputStream ips = null;
        ByteArrayOutputStream baout = new ByteArrayOutputStream();

        try {
            ips = new ByteArrayInputStream(content);
            

            if (fileName.endsWith(".jpg") || fileName.endsWith(".JPG") || fileName.endsWith(".jpeg") || fileName.endsWith(".JPEG")) {
                
                input = ImageIO.read(ips);

            } else if (fileName.endsWith(".gif") || fileName.endsWith(".GIF")) {

                input = ImageIO.read(ips);

            } else if (fileName.endsWith(".bmp") || fileName.endsWith(".BMP")) {

                // Wrap the InputStream in a SeekableStream.

                SeekableStream s = SeekableStream.wrapInputStream(ips, false);

                // Create the ParameterBlock and add the SeekableStream to it.
                ParameterBlock pb = new ParameterBlock();
                pb.add(s);

                // Perform the BMP operation
                RenderedOp img1 = JAI.create("BMP", pb);

                input = getBufferedImage(img1.getAsBufferedImage());
            } else if (fileName.endsWith(".png") || fileName.endsWith(".PNG")) {

                SeekableStream s = SeekableStream.wrapInputStream(ips, false);

                // Create the ParameterBlock and add the SeekableStream to it.
                ParameterBlock pb = new ParameterBlock();
                pb.add(s);

                // Perform the PNG operation
                RenderedOp img1 = JAI.create("PNG", pb);

                input = getBufferedImage(img1.getAsBufferedImage());
            }

            if (input == null)
                return null;

            // Get Writer and set compression
            Iterator iter = ImageIO.getImageWritersByFormatName("jpg");

            if (iter.hasNext()) {

                ImageWriter writer = (ImageWriter) iter.next();
                ImageWriteParam iwp = writer.getDefaultWriteParam();
                iwp.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
                float values[] = iwp.getCompressionQualityValues();
                System.out.println("compression Quality >> "+values[2]);
                iwp.setCompressionQuality(0.2f);
                //BufferedImage image = ImageIO.read(ips);
                IIOImage image1 = new IIOImage(input, null, null);
                writer.setOutput(new MemoryCacheImageOutputStream(baout));
                // writer.write(null, new IIOImage(image, null, null), iwp);
                writer.write(null, image1, iwp);
                input.flush();
                writer.dispose();
                writer = null;
                image1 = null;
                input = null;

            }
        } catch (FileNotFoundException finfExcp) {
            System.out.println(finfExcp);
        } catch (IOException ioExcp) {
            System.out.println(ioExcp);
        }
        return baout.toByteArray();
    }

    private BufferedImage getBufferedImage(Image img) {

        // if the image is already a BufferedImage, cast and return it
        // if ((img instanceof BufferedImage)) {
        // return (BufferedImage) img;
        // }

        // otherwise, create a new BufferedImage and draw the original
        // image on it
        int w = img.getWidth(null);
        int h = img.getHeight(null);
        int thumbWidth = 330;
        int thumbHeight = 250;

        // if width is less than 330 keep the width as it is.
        if (w < thumbWidth)
            thumbWidth = w;

        // if height is less than 250 keep the height as it is.
        if (h < thumbHeight)
            thumbHeight = h;

        // if less than 330*250 then do not compress
        if (w > 330 || h > 250) {

            double imageRatio = (double) w / (double) h;
            double thumbRatio = (double) thumbWidth / (double) thumbWidth;

            if (thumbRatio < imageRatio) {
                thumbHeight = (int) (thumbWidth / imageRatio);
            } else {
                thumbWidth = (int) (thumbHeight * imageRatio);
            }
        }
        // draw original image to thumbnail image object and
        // scale it to the new size on-the-fly
        BufferedImage bi = new BufferedImage(thumbWidth, thumbHeight,
                BufferedImage.TYPE_INT_RGB);
        Graphics2D g2d = bi.createGraphics();
        g2d.drawImage(img, 0, 0, thumbWidth, thumbHeight, null);
        g2d.dispose();
        return bi;
    }
    
}

Now in your Upload Action, write below code

for(FileBean fileBean: fileList){
                    if(fileBean.getActualFile()!=null && fileBean.getActualFile().getFileSize()>0)
                    {
                        //byte[] bFile = new byte[ fileBean.getActualFile().getFileSize()];    
                        
                        byte[] bFile = fileBean.getFileContent();    
                        
                        
                        fileBean.getActualFile().getInputStream().read(bFile);        
                        MyDocumentImage mydocimage=new MyDocumentImage();
                        //MyDocumentImageId mydocimageId=new MyDocumentImageId();
                        mydocimage.setMyDocImage(new MyDocImageType(fileType));
                        
                        if (fileBean.getActualFile().getFileName().endsWith(".jpg") || fileBean.getActualFile().getFileName().endsWith(".JPG") || fileBean.getActualFile().getFileName().endsWith(".jpeg") || fileBean.getActualFile().getFileName().endsWith(".JPEG") || fileBean.getActualFile().getFileName().endsWith(".gif") || fileBean.getActualFile().getFileName().endsWith(".GIF") || fileBean.getActualFile().getFileName().endsWith(".bmp") || fileBean.getActualFile().getFileName().endsWith(".BMP") || fileBean.getActualFile().getFileName().endsWith(".png") || fileBean.getActualFile().getFileName().endsWith(".PNG") && fileBean.getActualFile().getFileSize()>300000)
                        {
                            mydocimage.setDocContent(img.compressFile(bFile, fileBean.getActualFile().getFileName()));
                        }
                        else
                        {
                            mydocimage.setDocContent(bFile);
                        }
                            
                        mydocimage.setDocCreatedBy(userId);
                        mydocimage.setDocCreatedDate(new Timestamp(System.currentTimeMillis()));
                        mydocimage.setSdocLatest(true);
                        mydocimage.setSdocMimeType(fileBean.getActualFile().getContentType());
                        mydocimage.setSdocFileName(fileBean.getActualFile().getFileName());
                        mydocimage.setDocActive("Y");
                        mydocimage.setDocVersion(1);
                        //String docSeq = commonDAO.getNextSequence("my_doc.seq");
                        //mydocimage.setSCode(Integer.valueOf(sdocSeq));
                        //mydocimage.setSdocDtlCode(fileType);
                        //mydocimage.setSMewCode(staStall.getId().getStaBusCode());
                        //mydocimage.setSCode(staStall.getId().getStaCode());
                        //mydocimage.setId(mydocimageId);
                        mydocimage.setDs(dStp);

                        myDocList.getMyDocumentImages().add(mydocimage);
                    }
                }

Thats it, above code will compress images of particular size. Email me if you need source code or guidance on this.

Cheers,

Ujjwal Soni

Multiple skype instances on MACBOOK

Hi,

I have two skype id's and i had to login both of them simentaneously, so i opened up terminal and executed below command.
 

sudo /Applications/Skype.app/Contents/MacOS/Skype /secondary 

Cheers,

Ujjwal Soni