java - garbage collector does not work -
i have program download stocks data internet , create stock objects contains stock meta data , array of historical prices.
at first ran on more 20000 stocks created them , entered them arraylist writing them db(all of them in 1 transaction). wasn't idea , before ended download stocks wanted program died because of outofmemory.
then decided after every 500 stocks adding arraylist write them db , clean arraylist(arraylist.clear() gc make it's "magic") , fill arraylist new 500 stocks , go through same proccess on again.
that didn't work either, , program died again because of outofmemory exception.
i thought problem may in else in code , made experiment , ran same code 1 little difference: after create each stock object not put arraylist , continue , create stock objects on again without adding them arraylist.
the result program didn't consume memory @ , make me confused , frustrated.
please me find out if wrong program.
here lines of code:
the first version:
arraylist<stock> stocksdata = new arraylist<stock>(); stock stock; bufferedreader br = null; string line = ""; try { br = new bufferedreader(new filereader(yahoo_stocks_symbols)); while ((line = br.readline()) != null) { string[] stockmetadata = line.split(csv_split); stock = new stock(); if (stockmetadata.length >= 4) { stock.setsymbol(stockmetadata[0]); stock.setname(stockmetadata[1]); stock.setexchange(stockmetadata[2]); stock.setcategory(stockmetadata[3]); data_updater.updatestockdata(stock, fromdate, todate); if (stock.gethistoricaldata() != null && stock.gethistoricaldata().size() > 0) { stocksdata.add(stock); } } } }
the second version:
arraylist<stock> stocksdata = new arraylist<stock>(); stock stock; bufferedreader br = null; string line = ""; try { br = new bufferedreader(new filereader(yahoo_stocks_symbols)); while ((line = br.readline()) != null) { string[] stockmetadata = line.split(csv_split); stock = new stock(); if (stockmetadata.length >= 4) { stock.setsymbol(stockmetadata[0]); stock.setname(stockmetadata[1]); stock.setexchange(stockmetadata[2]); stock.setcategory(stockmetadata[3]); data_updater.updatestockdata(stock, fromdate, todate); if (stock.gethistoricaldata() != null && stock.gethistoricaldata().size() > 0) { stocksdata.add(stock); } } if (stocksdata.size() == 500) { writingutils.getinstance().updatestocks(stocksdata); stocksdata.clear(); // tried //system.gc(); } } }
im adding more information:
stock fields:
protected string _symbol; protected string _name; protected string _exchange; protected date _uptodate; protected arraylist<dailydata> _historicaldata;
dailydata fields:
private date _date; private double _open; private double _close; private double _adjclose; private double _high; private double _low; private double _volume;
first start profiling application tool jvisualvm (which comes jdk) , pinpointing objects being retained in heap.
second, there no 'clean()
' method on arraylist
, not clear did prevent list growing indefinitely.
lastly, consider setting conditional breakpoint in program 'catch' when list growing beyond expectation.
edit: if you're asking :
'can outofmemoryerror due fact that:
protected arraylist<dailydata> _historicaldata;
is growing , contributing memory pressure?'
the answer yes. have tried increase memory process? try running:
java -xms1g -xmx1g *yourprogram*
to see if still fails. may not have leak, may application needs more memory doing.