multithreading - Python - Multithreaded Proxy Tester -

-

i'm building proxy checker using multithreads, specificly thread pool from:

from multiprocessing.dummy import pool threadpool.

the http request using urllib2.

what want each proxy run 20 requests. if 1 threaded take time. thats multithreads power comes help. once set proxy want run 20 requests, , manage 2 things. 1 count exceptions , dump proxy if many occurs. 2nd save average response time , present later.

i don't manage implement above. have implemented 1 thread:

import socket import ssl import time import urllib import urllib2 import httplib  proxylist = []  def loadproxysfromfile(filename):     global proxylist     open(filename) f:         proxylist = [line.rstrip('\n') line in f]  def seturllib2proxy(proxyaddress):     proxy = urllib2.proxyhandler({         'http': "http://" + proxyaddress,         'https': "https://" + proxyaddress     })     opener = urllib2.build_opener(proxy)     urllib2.install_opener(opener)  def timingrequest(proxy, url):     error = false     seturllib2proxy(proxy)      start = time.time()     try:         req = urllib2.request(url)         urllib2.urlopen(req, timeout=5)     #opening request (getting response)      except (urllib2.urlerror, httplib.badstatusline, ssl.sslerror, socket.error) e:         error = true      end = time.time()     timing = end - start     if error:         print "error proxy " + proxy         return 0     else:         print proxy + " request " + url + " took: %s"  %timing + " seconds."         return timing  # main loadproxysfromfile("proxylist.txt") proxy in proxylist:     print "testing: " + proxy print "\n"  request_num = 20 error_tolerance_num = 3 resultlist = []  proxy in proxylist:     avgtime = 0     errorcount = 0     x in range(0, request_num):         result = timingrequest(proxy, 'https://www.google.com')         if (result == 0):             errorcount += 1             if (errorcount >= error_tolerance_num):                 break         else:             avgtime += result     if (errorcount < error_tolerance_num):         avgtime = avgtime/(request_num-errorcount)         resultlist.append(proxy + " has average response time of: %s" %avgtime)  print '\n' print "results summery: " print "-----------------" res in resultlist:     print res

things must done are:

for every proxy: wait until 20 requests on before changing proxy. sync somehow threads when adding calculate average response time (includes not take in account exceptions)

the best solutions i've read far using from multiprocessing.dummy import pool threadpool , pool.map(func, iterable) cant figure out how implement in code.


Comments

Post a Comment

Popular posts from this blog

python - TypeError: start must be a integer -

-
i trying draw 5 turtles, getting typeerror on line 25. here code: import turtle wn = turtle.screen() redrose = turtle.turtle() color = input("what background color be?") fillcolor_f = input("what color of rose be?") redrose.hideturtle() redrose.speed(30) redrose.penup() redrose.left(180) redrose.forward(175) redrose.right(90) redrose.forward(30) redrose.right(90) redrose.pendown() def drawrose(red): redrose.color("pink") redrose.fillcolor(fillcolor_f) redrose.fill(true) in range(red): redrose.forward(i) redrose.right(49) in range(5): drawrose(redrose) redrose.penup() redrose.forward(350) redrose.right(144) redrose.pendown() redrose.fill(false) drawrose(50) wn.bgcolor(color) i trying draw 5 roses, produces errors. doing in interactivepython.org. you recursively calling drawrose wrong parameter. on line 23 ( for in range(red): ) expect red integer...
Read more

c# - DevExpress RepositoryItemComboBox BackColor property ignored -

-
i have following code almost works properly; private void cbstatus_drawitem(object sender, listboxdrawitemeventargs e) { string item = e.item string; if (item != null) { switch (item) { case "1": e.appearance.forecolor = color.green; e.appearance.backcolor = color.green; break; case "2": e.appearance.forecolor = color.orange; e.appearance.backcolor = color.orange; break; case "3": e.appearance.forecolor = color.red; e.appearance.backcolor = color.red; break; } } } when dropdown shown, items forecolor correct, backcolor remains whatever backcolor of theme is; i.e. if i've got set dark theme, backcolor dark, cells in gridview, rather being green/orange/red. i've tried setting e.appearance.options.usebackcolor trying set e....
Read more

django - Creating multiple model instances in DRF3 -

-
1) create multiple model instances 1 api call, asked here: how create multiple model instances django rest framework? tried solution named in link, no success. i trying upload multiple files in 1 api call. result: files uploaded (only once overwrote perform_create) 1 instance created (if send 2 files, latter created instance). my code: class fileuploadserializer(serializers.modelserializer): def __init__(self, *args, **kwargs): many = kwargs.pop('many', true) super(fileuploadserializer, self).__init__(many=many, *args, **kwargs) class meta: model = fileupload read_only_fields = ('created', 'datafile', 'owner') class fileuploadviewset(viewsets.modelviewset): queryset = fileupload.objects.all() serializer_class = fileuploadserializer parser_classes = (multipartparser, formparser, ) def perform_create(self, serializer): file_list = self.request.data.getlist('file') ...
Read more