google app engine - How to read/write (String and Array like) variables during Mapreduce iteration? -

-

i trying process large amounts of data of order 5-10 million.

i running mapper in googleappengine/java task rate of 100/s , bucket-size of 100 billing enabled.

reading , writing datastore during map iteration affects overall speed very large extent. if can read/write string , array variables other simple counters, speed things large extent.

background: trying dedupe large data respect multiple text fields. must run n map jobs , compare rest of data.


Comments

Post a Comment

Popular posts from this blog

linux - Using a Cron Job to check if my mod_wsgi / apache server is running and restart -

-
my group , running server based upon django , uses mod_wsgi run apache server. not working on project after over, attempting set cronjob similar functionality check if apache server has shut down(system restart or power failure), , if has, restart server me. i've found documentation on how check if apache server down , restart server if is, our server uses https , our start command pretty verbose. can use functionality provided in these examples: https://askubuntu.com/questions/277389/cron-job-to-restart-apache https://www.digitalocean.com/community/tutorials/how-to-use-a-simple-bash-script-to-restart-server-programs or need more complicated process make happen? the command use start server is python manage.py runmodwsgi --host 0.0.0.0 --port 8001 --https-port 8000 --ssl-certificate (certificate location) --server-name (domain name) i'm pretty new linux , using both mod-wsgi apache appreciated. you better off using --setup-only option mod_wsgi-expres...
Read more

actionscript 3 - TweenLite does not work with object -

-
i got following problem: i have object called tempscore game. this object blitted canvas renderer via copypixels method. object not display object. it's score-object (self made). score-object extends object called basicblitarrayobject. basicblitarrayobject extends eventdispatcher (therefore no display object). i tried apply several different tweenlite-plugins tempscore-object (i.e. transformaroundcenter, colormatrixfilter, etc.). nothing happens. absolutely nothing. sometimes error messages (when plugin requires display object , object not display object). far good. according greensock (maker of tweenlite) engine can tween numeric property of object. when plugin transformaroundcenter requires display object tweening have modify plugin working non-display object (tempscore). can't because it's way hard me. my game rests upon code: http://www.8bitrocket.com/book/ch11_blastermines.zip try apply tweenlite object called tempmine inside game class blastermines. ...
Read more

jQuery Ajax Render Fragments OR Whole Page -

-
i'm in throes of building application wraps mess of legacy code. page working on, need substitute fragment divs constructed on back-end -- or else need replace entire page altogether. idea there dynamically controlled flow needs satisfied before can forward legacy product. substituting fragments works fine, seen in below my_body_content swap. trouble comes when i'm trying render not fragment, whole page, in "body" swap. @ point page goes blank. i want similar errors returned server. want nice rest 404 error messages displayed on screen, legacy-product 404s show in legacy-product 404 page. yes, requirements project weird. that's not problem can fix. here's jquery invocation, names changed protect guilty: $.ajax({ url: "places/things", type: "post", data: json.stringify(somebadassobject), datatype: "text", accepts: "text/html", contenttype: "application/json; charset=...
Read more