python - Optimizing py2neo's cypher insertion -

-

i using py2neo import several hundred thousand nodes. i've created defaultdict map neighborhoods cities. 1 motivation more efficiently import these relationships having been unsuccessful neo4j's load tool.

because batch documentation suggests avoid using it, veered away implementation op of this post. instead documentation suggests use cypher. however, being able create nodes defaultdict have created. plus, found difficult importing information first link demonstrates.

to reduce speed of import, should create cypher transaction (and submit every 10,00) instead of following loop?

for city_name, neighborhood_names in city_neighborhood_map.iteritems():      city_node = graph.find_one(label="city", property_key="name", property_value=city_name)          neighborhood_name in neighborhood_names:               neighborhood_node = node("neighborhood", name=neighborhood_name)               rel = relationship(neighborhood_node, "in", city_node)               graph.create(rel)

i time-out, , appears pretty slow when following. when break transaction commits every 1,000 neighborhoods, still processes slowly.

tx = graph.cypher.begin() statement = "merge (city {name:{city_name}}) create (neighborhood { name : {neighborhood_name}}) create (neighborhood)-[:in]->(city)" city_name, neighborhood_names in city_neighborhood_map.iteritems():     neighborhood_name in neighborhood_names:         tx.append(statement, {"city_name": city_name, "neighborhood_name": neighborhood_name}) tx.commit()

it fantastic save pointers each city don't need each time merge.

it may faster in 2 runs, i.e. create nodes first unique constraints (which should fast) , create relationships in second round.

constraints first, use labels city , neighborhood, faster match later:

graph.schema.create_uniqueness_constraint('city', 'name') graph.schema.create_uniqueness_constraint('neighborhood', 'name')

create nodes:

tx = graph.cypher.begin()  statement = "create (:city {name: {name}})" city_name in city_neighborhood_map.keys():     tx.append(statement, {"name": city_name})  statement = "create (:neighborhood {name: {name}})" neighborhood_name in neighborhood_names: # neighborhood names     tx.append(statement, {name: neighborhood_name})  tx.commit()

relationships should fast (fast match due constraints/index):

tx = graph.cypher.begin() statement = "match (city:city {name: {city_name}}), match (n:neighborhood {name: {neighborhood_name}}) create (n)-[:in]->(city)" city_name, neighborhood_names in city_neighborhood_map.iteritems():     neighborhood_name in neighborhood_names:         tx.append(statement, {"city_name": city_name, "neighborhood_name": neighborhood_name})  tx.commit()

Comments

Post a Comment

Popular posts from this blog

python - TypeError: start must be a integer -

-
i trying draw 5 turtles, getting typeerror on line 25. here code: import turtle wn = turtle.screen() redrose = turtle.turtle() color = input("what background color be?") fillcolor_f = input("what color of rose be?") redrose.hideturtle() redrose.speed(30) redrose.penup() redrose.left(180) redrose.forward(175) redrose.right(90) redrose.forward(30) redrose.right(90) redrose.pendown() def drawrose(red): redrose.color("pink") redrose.fillcolor(fillcolor_f) redrose.fill(true) in range(red): redrose.forward(i) redrose.right(49) in range(5): drawrose(redrose) redrose.penup() redrose.forward(350) redrose.right(144) redrose.pendown() redrose.fill(false) drawrose(50) wn.bgcolor(color) i trying draw 5 roses, produces errors. doing in interactivepython.org. you recursively calling drawrose wrong parameter. on line 23 ( for in range(red): ) expect red integer...
Read more

c# - DevExpress RepositoryItemComboBox BackColor property ignored -

-
i have following code almost works properly; private void cbstatus_drawitem(object sender, listboxdrawitemeventargs e) { string item = e.item string; if (item != null) { switch (item) { case "1": e.appearance.forecolor = color.green; e.appearance.backcolor = color.green; break; case "2": e.appearance.forecolor = color.orange; e.appearance.backcolor = color.orange; break; case "3": e.appearance.forecolor = color.red; e.appearance.backcolor = color.red; break; } } } when dropdown shown, items forecolor correct, backcolor remains whatever backcolor of theme is; i.e. if i've got set dark theme, backcolor dark, cells in gridview, rather being green/orange/red. i've tried setting e.appearance.options.usebackcolor trying set e....
Read more

django - Creating multiple model instances in DRF3 -

-
1) create multiple model instances 1 api call, asked here: how create multiple model instances django rest framework? tried solution named in link, no success. i trying upload multiple files in 1 api call. result: files uploaded (only once overwrote perform_create) 1 instance created (if send 2 files, latter created instance). my code: class fileuploadserializer(serializers.modelserializer): def __init__(self, *args, **kwargs): many = kwargs.pop('many', true) super(fileuploadserializer, self).__init__(many=many, *args, **kwargs) class meta: model = fileupload read_only_fields = ('created', 'datafile', 'owner') class fileuploadviewset(viewsets.modelviewset): queryset = fileupload.objects.all() serializer_class = fileuploadserializer parser_classes = (multipartparser, formparser, ) def perform_create(self, serializer): file_list = self.request.data.getlist('file') ...
Read more