java - Handling very large amount of data in MyBatis -

my goal dump data of database xml file. database not terribly big, it's 300mb. problem have memory limitation of 256mb (in jvm) only. cannot read memory.

i managed solve problem using ibatis (yes mean ibatis, not mybatis) calling it's getlist(... int skip, int max) multiple times, incremented skip. solve memory problem, i'm not impressed speed. variable names suggests method under hood read entire result-set skip specified record. sounds quite redundant me (i'm not saying that's method doing, i'm guessing base on variable name).

now, switched mybatis 3 next version of application. question is: there better way handle large amount of data chunk chunk in mybatis? there anyway make mybatis process first n records, return them caller while keeping result set connection open next time user calls getlist(...) start reading n+1 record without doing "skipping"?

no, mybatis not have full capability stream results yet.

edit 1: if don't need nested result mappings implement custom result handler stream results. on current released versions of mybatis. (3.1.1) current limitation when need complex result mapping. nestedresultsethandler not allow custom result handlers. fix available, , looks targeted 3.2. see issue 577.

in summary, stream large result sets using mybatis you'll need.

1. implement own resultsethandler.
2. increase fetch size. (as noted below guillaume perrot)
3. for nested result maps, use fix discussed on issue 577. fix resolves memory issues large result sets.