Commit0c73d02

committed

- Added activerecord-import gem to bulk import

- Added activerecord transaction and used bulk import of data content

1 parente93d8a0 commit0c73d02Copy full SHA for 0c73d02

File tree

+14

-10

lines changed

+14

-10

lines changed

Lines changed: 1 addition & 0 deletions

Original file line number	Diff line number	Diff line change
`@@ -12,6 +12,7 @@ gem 'rails', '~> 5.2.0'`
`12`	`12`	`gem'mysql2','~> 0.5.1'`
`13`	`13`	`# Use Puma as the app server`
`14`	`14`	`gem'puma','~> 3.7'`
	`15`	`+gem'activerecord-import','~> 0.23.0'`
`15`	`16`	`gem'jsonapi-resources','~> 0.9.0'`
`16`	`17`	`gem'httparty','~> 0.16.2'`
`17`	`18`	`# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder`

Lines changed: 3 additions & 0 deletions

Lines changed: 10 additions & 10 deletions

Original file line number	Diff line number	Diff line change
`@@ -7,24 +7,24 @@ def parse`
`7`	`7`	`response=HTTParty.get(params[:ref_url])`
`8`	`8`	`doc=Nokogiri::HTML(response.body)`
`9`	`9`
	`10`	`+contents=[]`
`10`	`11`	`%w[h1h2h3a].eachdo \|tag\|`
`11`	`12`	`doc.xpath("//#{tag}").eachdo \|element\|`
`12`		`-extract_element(element,tag)`
	`13`	`+content=Content.new`
	`14`	`+content.url=params[:ref_url]`
	`15`	`+content.tag=tag`
	`16`	`+content.content=element.text`
	`17`	`+contents <<content`
`13`	`18`	`end`
`14`	`19`	`end`
`15`	`20`
	`21`	`+Content.transactiondo`
	`22`	`+Content.import!contents,validate:false`
	`23`	`+end`
	`24`	`+`
`16`	`25`	`renderjson:{success:true,code:response.code,message:response.message}`
`17`	`26`	`rescueException=>e`
`18`	`27`	`renderjson:e`
`19`	`28`	`end`
`20`	`29`	`end`
`21`		`-`
`22`		`-private`
`23`		`-defextract_element(element,tag='')`
`24`		`-content=Content.new`
`25`		`-content.url=params[:ref_url]`
`26`		`-content.tag=tag`
`27`		`-content.content=element.text`
`28`		`-content.save!`
`29`		`-end`
`30`	`30`	`end`

Comments

(0)