Webpage screen capturing using khtml2png

Recently, we were working on one PHP project, where we required to have "webpage screen capturing" functionality. I googled on net and found some tools... some window based, some paid... obviously I was looking for *FREE* tool :). As we are working on Lamp (Linux, Apache, MySQL & PHP), I was wondering if I get some Linux based tool.

One solution I found was, using html2ps and then ps2png/ps2jpg/ps2gif to convert it to image. Then ImageMagicK for image manipulation. Somehow I stuck with some weird memory related errors, some package conflicts, some formatting issues etc. So, after spending one day on nothing; I dropped this solution.

Then I tried khtml2png (http://khtml2png.sourceforge.net) and after some r&d, it worked for us.

Some points to remember...
- You need to have VPS/dedicated hosting to setup these tools. On shared hosting, its not possible to install due to various restrictions by hosting providers.

- This tool requires, some libraries and tools: g++, KDE 3.x, kdelibs for KDE 3.x, zlib (zlib1g-dev) and cmake

- This tool uses KDE (K Desktop Environment), that means whenever you use khtml2png tool, it will open one window for *a while* at time of capturing webpage screenshot. We can remove this by using "Xvfb". We will see how to install and configure it later.

- These links will be helpful, if you are planning to develop web application with webpage screen capturing using khtml2png
http://khtml2png.sourceforge.net/index.php?page=faq
http://www.mysql-apache-php.com/website_screenshot.htm

Here is step by step guide to install various dependencies and packages. (I installed these tools on Fedora7 & RHEL5 successfully)

I used "yum" command to install and auto-configure these tools. If "yum" is not available on your machine, get if from http://yum.baseurl.org/ and install it.

Step:1

 yum install ImageMagick

 yum install Xvfb

 yum install gcc gcc-c++ automake autoconf nano zlib zlib-devel

 yum groupinstall "X Window System" "KDE (K Desktop Environment)"

 yum install kdelibs kdelibs-devel

 yum install Xvfb xorg xorg-x11-font* 

Step:2 Install *cmake*
Go to share directory by typing command

cd /usr/local/share/

or any preferred directory where you want to download package. (check http://www.cmake.org for latest "cmake" version)

 wget http://www.cmake.org/files/v2.8/cmake-2.8.1.tar.gz
     
 tar -xzvf cmake-2.8.1.tar.gz

 cd cmake-2.8.1

 ./bootstrap

 make

 make install

Step:3 Download & Install *khtml2png* on your server as per instructions in this link.
http://khtml2png.sourceforge.net/index.php?page=download

Step:4 Check if *khtml2png* is working

 /usr/local/bin/khtml2png2 'http://www.yahoo.com' yahoo.png 

(this will capture yahoo homepage in yahoo.png)

Step:5 Install *khtmld* (a daemon which will be required to run khtml2png in background)
http://wiki.goatpr0n.de/projects/khtmld

I faced couple of problems while setting up *khtmld*, but it got solved by reading suggestions from above link.

I installed above all tools as *root* user.

Once you are done with above steps, lets play with *khtml2png*

How to start?
Run following command to run khtml2png without a visible X session

 Xvfb :2 -screen 0 1024x768x24&
 export DISPLAY=localhost:2.0

(you can put above 2 lines in rc.local so it will start automatically whenever server restarts)

Then start *khtmld* daemon as your webserver user (for me it is *apache*) so that PHP script can have permission to talk with this daemon. (run below command after login as *root* user)

 khtmld -K /usr/local/bin/khtml2png2 -c /etc/khtmldrc --user apache&

"-K /usr/local/bin/khtml2png2" is path to khtml2png2 as by default "khtmld" will look for old "khtml2png" (khtml2png2 is latest version). Find khtml2png2 path using

 whereis khtml2png2

"-c /etc/khtmldrc" is config file path for khtmld (you can create this config file if its not already there)
Sample content for khtmldrc

width=1024
height=768
display=:0.0

Capture image using *khtmld*

echo "http://www.yahoo.com /tmp/yahoo.png" >/tmp/khtmldspool 

(for more details - http://wiki.goatpr0n.de/projects/khtmld)

We have also used ImageMagicK command "convert" (http://www.imagemagick.org/script/convert.php) to trim the image for removing whitespace.

convert /tmp/yahoo.png -fuzz 1% -trim /tmp/new.yahoo.png

Sample PHP code for capturing & displaying PNG image using "khtml2png"

<?php
  ob_clean();
  header("Cache-Control: no-cache");
  header("Pragma: no-cache");
  header("Content-type: image/png");

  $webpage_url= "http://www.yahoo.com";

  $out_put_file = "/tmp/yahoo.png"; //captured screen
  $new_out_put_file = "/tmp/new.yahoo.png"; //whitespace removed

  $cmd = "echo '".$webpage_url." ".$out_put_file."' >/tmp/khtmldspool";
  exec("$cmd");

  // some delay till khtml2png capture screen
  while(!file_exists($out_put_file)) { sleep(3); }

  exec("convert $out_put_file -fuzz 1% -trim $new_out_put_file");
 
  while(!file_exists($new_out_put_file)) { sleep(1); }

  // display image on browser
  echo file_get_contents($new_out_put_file);
        
  unlink($out_put_file);
  unlink($new_out_put_file);
  exit;
?>

Hope this will be helpful.

That's all for now.

Is losing a job is terrible experience? Turn it into an opportunity!

In last couple of months, we have heard lots of job cuts/layoffs in various field. According to Techcrunch, 80,076 job eliminations have been announced or completed since the beginning of the year in the tech industry. The total number of tech layoffs since late August is now 195,856. The number is big and is growing fast.



No one likes to be told to leave the job. But the reality is that it happens every day in the corporate world more so in the bad economy. Those who are asked to pack up aren’t always non-performers. There are several reasons behind it!
(a) company is not performing well.
(b) employees are overqualified
(c) their position has become redundant because the company does not want to pursue the project/business that they were involved in etc.

When I was working with my previous employer, I got a chance to have a quick look of this situation. See some example below.

(a) I was a part of technical team while interviewing a Sr. Developer. There were two positions open and the person was hired with high expectation. Considering seniority of the hired person, second position was closed. The same person got fired after 4 months due to tech layoffs started at company. The clear reason I see here is; "The person failed to perform his duty well or the expectations were too high for him"

(b) two or three employees got best performance award and those too got laid off. I still don’t understand why they laid off.

(c) So many senior persons from middle management resigned. What could be the reason? "Fear of being laid off?"

(d) Some senior persons from top management got fired. Clear reason - "Cost cutting / changes in management to apply new strategies"

Those who got fired were either fresher, experienced persons - who recently joined company or top management executives.

Apart from this if good employees get laid off; means company is really not performing well and cash flow is not enough to survive.

The firing news can hit you hard. Shock and anger are usual feelings, however in some cases it can lead to depression too. One of my colleague had postponed his marriage due to layoff news. How do you cope up with such negative feelings? How do you tell yourself and the world that you are a capable professional? That you have what it takes to be successful.

Some suggestions to handle this situation.

(a) Don't Blame Yourself. Have confidence in yourself. See the job loss as an opportunity to make a new beginning. Job is part of life; but life is not part of job.

(b) Always have a network of good industry friends/contacts. It is this network that you can tap into when you suddenly find yourself on road with no job in hands.

(c) Manage Finances Well.

(d) Even if you are safe; stay in touch with your work technology. This will help you to survive in bad time.

(e) Talk to your HR to help you in finding new job. At our company, management gave good compensation to employees who got laid off and also, they made arrangement with job consultant to help who got fired. You may feel so angry and upset that you don't want to take any help whatsoever from the same employer who gave you the pink slip at the first place, but relationship matters.

(f) Lots of people are there who believe that "You are fired means you are not performing well", but I see - scenario is changed now. People understand that this is not the only reason why employee gets fired. After layoff at our company I interviewed two employees who got laid off recently from another well-known IT company. One of them got selected after interview due to his capabilities. So, be sincere and honest with the prospective employers about your last job. It's always better that they hear the truth from you than from someone else.

Have confidence in your abilities and move on.