Shedding Those Excess Bytes

Scanned documents are notoriously large in size and take a long time to transmit by fax or the Internet. But new image compression software from CVISION Technologies is overcoming the "World Wide Wait" by throwing off excess bytes in a big way.

DESPITE the computer and Internet revolution, we are still surrounded by paper. Huge swathes of it in some cases. Take the law field. During a big trial there are normally several law firms working on the same case, some representing, some opposing. If it is a case against a large company, ten different states could be co-plaintiffs. All would need to share information. And that information - known as "discovery" information - could easily run into ten million pages, especially if the case lasts a number of years.

Once collected, discovery information is stored in web-based repositories that can be accessed by all involved in a case. The problem is, once they have been scanned, discovery documents can run into megabytes and take a very long time to transmit - besides hogging digital storage space. The answer is to compress the files for transmission and storage. One firm at the forefront of image compression technology is CVISION Technologies, based in the Borough of Queens in New York City (not far from DCL).

"A lot of law firms use our software so remote users can quickly download image files," says Ari Gross, Chief Technology Officer and founder of CVISION Technologies. "ASPs use us because they're in the business of hosting their clients' documents and can offer a competitive edge if documents can be retrieved faster."

Reducing the World Wide Wait
CVISION sees itself as helping the business world overcome what has been dubbed the "World Wide Wait." Many firms and organizations around the world still have 56 KB/sec dial-up connections to the Internet. Which means that even a standard PDF file can take quite some time to transmit over the web.

"With the latest JBIG2 compression technology, our PdfCompressor will shrink a 10-page full-color PDF brochure from 7,200 KBytes to 825KBytes," explains Gross. "That makes a lot of difference to transmission times when you're running a 56 KB connection."

Faxes can also be compressed considerably using PdfCompressor. A six page fax of 210 KBytes can be reduced to 30 KBytes. This was something Gross and his colleagues experimented with in the late 1990s, during the early days of the firm.

"We took the Ken Starr report, which was 695,000 bytes when Congress released it," recalls Gross. "We printed it out and scanned it. All of a sudden it exploded in size to 14 megabytes! Using our compression tool, which at that time wasn't PDF based, we brought the file size back down to 600,000 bytes, making it practical to transmit it by fax or the Internet."

Analyzing patterns
Besides law, CVISION numbers customers from banking, finance, government, intelligence services, and the military - many of whom have adopted web repositories for the storage of information. CVISION's range of compression products are all part of the CVista Suite. Each can be licensed separately and be run on company servers or on desktop computers.

But how does the software work?

"Our products look for repetitive patterns and glyphs that are the same within a document," explains Gross. "If 95 instances of the letter 'a' are found on a page, for example, they are represented internally just once for each unique instance. Typically, we'll see 95 'a's and represent them 5 times - each with a subtle difference because we're extremely cautious about modifying data in the slightest way. Sophisticated algorithms analyze each portion of a page image to work out which of our many approaches will work best to compress the various segments of the image to ensure the smallest file size possible with no perceptible loss of image clarity."

Wireless solution
Wireless technology is another area where compression of information is critical. Handheld computer users, for example, often won't download e-mail attachments because they take so long to download due to limited bandwidth.

"In a military situation it can take precious minutes to send a wireless fax from one machine to another," says Gross. "Transmission rates are less than 2,000 bytes per second. So it can take thirty seconds to transmit a simple black and white fax and seven minutes for a grey scale or color fax. That can mean life or death in a battle situation."

CVISION's software is able to provide a 10:1 level of compression. So, in conjunction with the various types of fax hardware currently in development, the company looks set to make it viable to use handheld devices in both military and commercial situations.

Impact on business and commerce
Because CVISION's latest products are based on JBIG2 (Joint Bi-Level Image Group) document compression technology - which was developed by the ITU (International Telegraphic Union) and is not proprietary - a lot of companies are adopting the technology to manage their documents. A major U.S. bank, for example, is converting all its mortgage documents (about a million and a half images per day) using CVISION's JBIG2 PdfCompression technology.

"The key is having technology that is non-proprietary," says Gross. "That's the good thing about using a standard like JBIG2. The ITU has approved it as the next fax standard. The new generation of copiers and scanners will support it. Plus Adobe supports JBIG2 in their Acrobat Reader. So these are solutions that are having a big impact on business and commerce - shrinking files saves money in terms of transmission times and it saves on storage space."

DCLnews Editorial

Read more about technology and data conversion at DCL Library.