SQL Power Business Intelligence Productivity Tools

SQL Power Software Forum

SQL Power Software Forum

  [Search] Search   [Recent Topics] Recent Topics   [Hottest Topics] Hottest Topics   [Members]  Member Listing   [Groups] Back to home page  [Register] Register /  [Login] Login 
Out of Memory - Java Heap size error  XML
Forum Index » SQL Power DQguru Bugs
Author Message
lilihammer


Joined: 2007-06-07 14:18:48
Messages: 101
Offline

Situation: Running MatchMaker on a de-duping project. One munge process that uses the translate word transformation. The result table already contains 2mil rows.

Error - Java Heap Size error.

Running the munge process again, having deleted all rows from the results table there are no issues.

Jeff
SQL Power Developer
[Avatar]

Joined: 2007-06-27 18:31:33
Messages: 410
Offline

Hmm, were you expecting a large number of matches for this run?

-Jeff
mikegreenan


Joined: 2008-05-12 11:22:46
Messages: 1
Offline

I've got similair situation. 2 million records, quite a bit of duplicates - MSSQL 2000, 2gb RAM Dual Core - any advice would be much appreciated - e.g is it a memory problem.
Jeff
SQL Power Developer
[Avatar]

Joined: 2007-06-27 18:31:33
Messages: 410
Offline

Hello,

Looks like the Java Virtual Machine (JVM) wasn't allocated enough memory to perform the matching process. The Power*MatchMaker stores the match data in memory, so if you have a lot of duplicates, then it can fill up memory quickly.

You can read this thread to see our justification for doing most of the work in memory. http://www.sqlpower.ca/forum/posts/list/1827.page

You can increase the amount of memory that the VM uses by setting or altering the Java Virtual Machine arguments when you run Power*MatchMaker, which will depend on which OS you're running Power*MatchMaker on.

Since you're using MS SQL Server, I'm presuming you're using the Windows version (correct me if I'm wrong). In which case, you can increase the memory allocated to the JVM by:

1) Opening the properties for the Power*MatchMaker shortcut generated by the installer.

2) Then in the 'Target' field, there should be the command to run Power*MatchMaker. There should already be an argument that says '-Xmx600m'.

3) Change the '600' to the number of MB of memory that you'd like to allocate to the the JVM. (ex. '-Xmx1024m', or maybe even '-Xmx2048m').

4) Then try running that match again.


-Jeff
lilihammer


Joined: 2007-06-07 14:18:48
Messages: 101
Offline

Hey Jeff,

I would suggest that the default java heap size is larger.
Also, if the user doesn't have the short cut created, where would you go to change the java heap size error?
Jeff
SQL Power Developer
[Avatar]

Joined: 2007-06-27 18:31:33
Messages: 410
Offline

I'm not sure how practical it would be to set the Java heap size to anything significantly larger than 600m. It can cause the Power*MatchMaker to not even run.

If you tried setting the maximum heap size to 1024m (1GB) on a Windows XP machine for example, and ran it and you had only 1GB of system memory to begin with, then the application will not run at all. Instead you'd get an error saying that the VM couldn't start because it couldn't reserve enough memory.

As for altering the maximum heap size, it would depend on how you are starting up the Power*MatchMaker. If you're not using the shortcut, then how are you starting it up?

For example, if you were running it from the command-line, then you would run the application like so:



Change the 1024 number to however many MB of memory that you would like to reserve.

Or, you can make a custom shortcut instead, then you'll need to modify it.

Right-click on the shortcut, and choose 'Properties'. Then in the 'Target' field, it should contain the contents



What you need to replace it with is:



Again, changing the 1024 to whatever amount you actually need.

Hope this helps!

This message was edited 4 times. Last update was at 2008-08-26 16:28:16


-Jeff
 
Forum Index » SQL Power DQguru Bugs
Go to:   
Powered by JForum 2.1.8 © JForum Team