Migrate restore script to promises

[dusterio]

Tested on a table with 100,000 entries
Migrated to promises and ES6 syntax because it's supported in Node
The new logic is like this:
Every line of the standard input creates one promise (an insert task), then these promises are grouped in batches depending on Writing Capacity. Eg. if there are 50 writing units, there will be 50 launched promises in parallel. Then, batches will run one after another in series that will allow to introduce delays.

[0crat]

@yegor256 please, pay attention to this pull request

[0crat]

Job gh:yegor256/dynamo-archive#32 is in scope.

[0crat]

+15 points just awarded to @dusterio, total is +15.

[yegor256]

@rultor merge

[rultor]

@rultor merge

@yegor256 OK, I'll try to merge now. You can check the progress of the merge here

[rultor]

@rultor merge

@dusterio @yegor256 Oops, I failed. You can see the full log here (spent 2min)

| |   `-- \u001b[40m\u001b[33mto-utf8@0.0.1\u001b[39m\u001b[49m 
| +-- \u001b[40m\u001b[33mminimist@0.0.10\u001b[39m\u001b[49m 
| `-- \u001b[40m\u001b[33monce@1.3.3\u001b[39m\u001b[49m 
|   `-- \u001b[40m\u001b[33mwrappy@1.0.2\u001b[39m\u001b[49m 
+-- \u001b[40m\u001b[33mfaucet@0.0.1\u001b[39m\u001b[49m 
| +-- \u001b[40m\u001b[33mdefined@0.0.0\u001b[39m\u001b[49m 
| +-- \u001b[40m\u001b[33mduplexer@0.1.1\u001b[39m\u001b[49m 
| +-- \u001b[40m\u001b[33mminimist@0.0.5\u001b[39m\u001b[49m 
| +-- \u001b[40m\u001b[33msprintf@0.1.5\u001b[39m\u001b[49m 
| +-- \u001b[40m\u001b[33mtap-parser@0.4.3\u001b[39m\u001b[49m 
| | `-- \u001b[40m\u001b[33mreadable-stream@1.1.14\u001b[39m\u001b[49m 
| +-- \u001b[40m\u001b[33mtape@2.3.3\u001b[39m\u001b[49m 
| | +-- \u001b[40m\u001b[33mdeep-equal@0.1.2\u001b[39m\u001b[49m 
| | `-- \u001b[40m\u001b[33mjsonify@0.0.0\u001b[39m\u001b[49m 
| `-- \u001b[40m\u001b[33mthrough2@0.2.3\u001b[39m\u001b[49m 
|   +-- \u001b[40m\u001b[33mreadable-stream@1.1.14\u001b[39m\u001b[49m 
|   `-- \u001b[40m\u001b[33mxtend@2.1.2\u001b[39m\u001b[49m 
|     `-- \u001b[40m\u001b[33mobject-keys@0.4.0\u001b[39m\u001b[49m 
+-- \u001b[40m\u001b[33mminimist@1.1.3\u001b[39m\u001b[49m 
+-- \u001b[40m\u001b[33msleep@3.0.1\u001b[39m\u001b[49m 
| `-- \u001b[40m\u001b[33mnan@2.6.2\u001b[39m\u001b[49m 
`-- \u001b[40m\u001b[33mtape@2.13.4\u001b[39m\u001b[49m 
  +-- \u001b[40m\u001b[33mdeep-equal@0.2.2\u001b[39m\u001b[49m 
  +-- \u001b[40m\u001b[33mglob@3.2.11\u001b[39m\u001b[49m 
  | `-- \u001b[40m\u001b[33mminimatch@0.3.0\u001b[39m\u001b[49m 
  |   +-- \u001b[40m\u001b[33mlru-cache@2.7.3\u001b[39m\u001b[49m 
  |   `-- \u001b[40m\u001b[33msigmund@1.0.1\u001b[39m\u001b[49m 
  +-- \u001b[40m\u001b[33minherits@2.0.3\u001b[39m\u001b[49m 
  +-- \u001b[40m\u001b[33mobject-inspect@0.4.0\u001b[39m\u001b[49m 
  +-- \u001b[40m\u001b[33mresumer@0.0.0\u001b[39m\u001b[49m 
  `-- \u001b[40m\u001b[33mthrough@2.3.8\u001b[39m\u001b[49m 

\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m\u001b[35m\u001b[0m dynamo-archive@0.0.0 license should be a valid SPDX license expression
\u001b[0m\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35moptional\u001b[0m SKIPPING OPTIONAL DEPENDENCY: leveldown@0.10.6 (node_modules/leveldown):
\u001b[0m\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35moptional\u001b[0m SKIPPING OPTIONAL DEPENDENCY: leveldown@0.10.6 install: `node-gyp rebuild`
\u001b[0m\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[30;43mWARN\u001b[0m \u001b[0m\u001b[35moptional\u001b[0m SKIPPING OPTIONAL DEPENDENCY: Exit status 1
\u001b[0m+ npm test
\u001b[?25l[\u001b[90m..................\u001b[0m] - : \u001b[32minfo\u001b[0m \u001b[35musing\u001b[0m node@v6.11.1\u001b[K

\u001b[K\u001b[?25h
> dynamo-archive@0.0.0 test /home/r/repo
> tape test/archive.test.js | faucet


# Lo...\u001b[K

  ok \u001b[1m\u001b[32m1\u001b[0m Exit cleanly\u001b[K
  not ok \u001b[1m\u001b[31m2\u001b[0m Clean stdout\u001b[K
\u001b[2A\u001b[1G\u001b[1m\u001b[31m⨯ Lo...\u001b[0m\u001b[2B\u001b[1G    ---
      operator: notOk
      expected:
        false
      actual:
        'Total promises: 4\nProcessing 1 batches of promises, up to 10 promises each\nProcessed a batch with 4 items\nAll done, hooray!\n'
      at: maybeClose (internal/child_process.js:891:16)
    ...

  ok \u001b[1m\u001b[32m3\u001b[0m Clean stderr\u001b[K
# Expo...\u001b[K

  ok \u001b[1m\u001b[32m4\u001b[0m Exit cleanly\u001b[K
  ok \u001b[1m\u001b[32m5\u001b[0m Clean stderr\u001b[K
  ok \u001b[1m\u001b[32m6\u001b[0m Got results\u001b[K
  ok \u001b[1m\u001b[32m7\u001b[0m Found items\u001b[K
\u001b[1A\u001b[1G\u001b[1m\u001b[32m✓ Expo...\u001b[0m\u001b[1B\u001b[1G
# Export archive ...\u001b[K

  ok \u001b[1m\u001b[32m8\u001b[0m Exit cleanly\u001b[K
  ok \u001b[1m\u001b[32m9\u001b[0m Clean stderr\u001b[K
  ok \u001b[1m\u001b[32m10\u001b[0m Got results\u001b[K
  ok \u001b[1m\u001b[32m11\u001b[0m Found items\u001b[K
\u001b[1A\u001b[1G\u001b[1m\u001b[32m✓ Export archive ...\u001b[0m\u001b[1B\u001b[1G
...\u001b[K

...
...\u001b[K

...
...\u001b[K
\u001b[1A\u001b[1G\u001b[1m\u001b[31m...\u001b[0m\u001b[1B\u001b[1G\u001b[37;40mnpm\u001b[0m \u001b[0m\u001b[31;40mERR!\u001b[0m\u001b[35m\u001b[0m Test failed.  See above for more details.
\u001b[0mcontainer bb0d2edcc80b1e63a052c103533c22632cec202ee9244efd4093f020b4631798 is dead
Tue Aug  8 07:28:02 CEST 2017

[dusterio]

Weird - it runs just fine for me (both npm install and npm test) but it fails on Travis.

How/where is stuff like LevelUp, LevelDB, etc are used? It's some pretty heavy stuff for such a tiny tool :)

[0crat]

@yegor256/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually or invite more people to the project, as explained in §51

[0crat]

@yegor256/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually or invite more people to the project, as explained in §51

[yegor256]

@dusterio are you planning to finish this?

[0crat]

@yegor256/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually or invite more people to the project, as explained in §51

[0crat]

@yegor256/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)

[0crat]

@yegor256/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)

[yegor256]

@0crat out

[0crat]

@0crat out (here)

@yegor256 The job #32 is now out of scope

[yegor256]

@dusterio what is the story here? Are you going to finish?

[yegor256]

@dusterio let's finish this?