Movatterモバイル変換


[0]ホーム

URL:


Skip to content
DEV Community
Log in Create account

DEV Community

Nguyễn Hữu Hiếu
Nguyễn Hữu Hiếu

Posted on

Java Spring Boot: batch insert data

Problem

  • Inserting one is very boring => Want to insert many at once. This is for you

Solution

// ... your packageimportlombok.Builder;importlombok.Data;importjavax.persistence.Column;importjavax.persistence.Entity;importjavax.persistence.GeneratedValue;importjavax.persistence.Id;importjavax.persistence.SequenceGenerator;@Data@Entity(name="student")@BuilderpublicclassStudentEntity{@Id@GeneratedValue(generator="student_sequence")@SequenceGenerator(name="student_sequence",sequenceName="student_sequence")privatelongid;@ColumnprivateStringname;@ColumnprivateStringemail;}
Enter fullscreen modeExit fullscreen mode
  • Step 3. (optional) If you don't usespring.jpa.hibernate.ddl-auto=create => need to make student_sequence by hand
-- demo.student_sequence definitioncreatetablestudent_sequence(next_valbigint)engine=InnoDBinsertintostudent_sequencevalues(1)
Enter fullscreen modeExit fullscreen mode
  • Step 4. Enable batch_size
# apllication.propertiesspring.jpa.hibernate.ddl-auto=update# auto update/create column, table ...spring.jpa.properties.hibernate.jdbc.batch_size=5# insert 5 entity at once
Enter fullscreen modeExit fullscreen mode
  • Step 5. Run Test with 5000 entity
// ... your packageimportcom.hieunh1801.demo.entity.StudentEntity;importcom.hieunh1801.demo.repository.StudentRepository;importlombok.extern.slf4j.Slf4j;importorg.junit.jupiter.api.Test;importorg.springframework.beans.factory.annotation.Autowired;importorg.springframework.boot.test.context.SpringBootTest;importorg.springframework.util.StopWatch;importjava.util.ArrayList;importjava.util.List;@Slf4j@SpringBootTestclassDemoApplicationTests{@AutowiredprivateStudentRepositorystudentRepository;@TestvoidinsertStudentSeparately(){log.info("insert 5000 students separately");StopWatchwatch=newStopWatch();watch.start();for(inti=0;i<5000;i++){StudentEntitystudentEntity=this.createStudent(i);this.studentRepository.save(studentEntity);}watch.stop();log.info("end insert students separately in {} ms",watch.getTotalTimeMillis());// 16372 ms}@TestvoidinsertStudentsByBatch(){log.info("insert 5000 students by batch");StopWatchwatch=newStopWatch();watch.start();List<StudentEntity>students=newArrayList<>();for(inti=0;i<5000;i++){students.add(this.createStudent(i));}this.studentRepository.saveAll(students);watch.stop();log.info("end insert students by batch in {} ms",watch.getTotalTimeMillis());//  2880 ms}privateStudentEntitycreateStudent(Integerindex){returnStudentEntity.builder().name("Student Name"+index).email("Student Email"+index).build();}}
Enter fullscreen modeExit fullscreen mode
  • Step 6. OutOfMemory: if you save 100,000 entities at once then JPA will save it inpersistent context meaning save to RAM => of course out of memory. You need to flush it!!!
for(inti=0;i<5000;i++){students.add(this.createStudent(i));if(i%100==0){// save and flush every 100 entitythis.studentRepository.saveAllAndFlush(students);students.clear();}}this.studentRepository.saveAllAndFlush(students);
Enter fullscreen modeExit fullscreen mode

Top comments(0)

Subscribe
pic
Create template

Templates let you quickly answer FAQs or store snippets for re-use.

Dismiss

Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment'spermalink.

For further actions, you may consider blocking this person and/orreporting abuse

  • Location
    Việt Nam
  • Education
    MTA
  • Work
    SPMED
  • Joined

More fromNguyễn Hữu Hiếu

DEV Community

We're a place where coders share, stay up-to-date and grow their careers.

Log in Create account

[8]ページ先頭

©2009-2025 Movatter.jp