Merge pull request #7 from qa-dev/readme

readme and small fixes

Author: andrskom

README.md

@@ -1,25 +1,119 @@
 # ~~jsonwire-grid~~WebDriverGrid [![Build Status](https://travis-ci.org/qa-dev/jsonwire-grid.svg?branch=master)](https://travis-ci.org/qa-dev/jsonwire-grid)
-This is scalable Golang implementation of Selenium Grid (hub).
+This is high-performance scalable implementation of Selenium Grid (hub),
+###### What is Selenium-Grid?
+>Selenium-Grid allows you run your tests on different machines against different browsers in parallel. That is, running multiple tests at the same time against different machines running different browsers and operating systems. Essentially, Selenium-Grid support distributed test execution. It allows for running your tests in a distributed test execution environment.
+
+## Features
+* One session per one node, no more no less😺
+* Scaling grid-instances for fault-tolerance
+* Support and effective management over 9000 nodes, for parallel testing👹
+* Single entry point for all your test apps
+* Send metrics to [statsd](https://github.com/etsy/statsd)
+* Support on-demand nodes in Kubernetes cluster (Only if grid running in cluster)
+
 
 ## HowTo
-1. Install MySql
-1. Set in config.json your db connesction string `[username[:password]@][protocol[(address)]]/dbname[?param1=value1&...&paramN=valueN]`
-1. Run app
-#### Run binary file
-1. download binary file and
-1. `export CONFIG_PATH=./config.json`
-1. `./jsonwire-grid`
-
-### Run From Source
-#### Requirements
-* Go >= 1.8.1
-* [go-bindata](https://github.com/jteeuwen/go-bindata)
-1. `git clone https://github.com/qa-dev/jsonwire-grid .`
-1. `cd jsonwire-grid`
-1. `cp config-sample.json config.json`
-1. `make run`
-
-## HowToUse
-1. Run app
+### Run grid
+1. [Download last release](https://github.com/qa-dev/jsonwire-grid/releases) and unzip
+1. cd to `jsonwire-grid_vXXX`
+1. Type `export CONFIG_PATH=./config-local-sample.json`
+1. Type `./jsonwire-grid`
+1. Grid running!
+
+### Run nodes
+1. [Download selenium](http://www.seleniumhq.org/download/)
 1. `java -jar selenium-server-standalone-3.4.0.jar -role node  -hub http://127.0.0.1:4444/grid/register`
-1. try create session `curl -X POST http://127.0.0.1:4444/wd/hub/session -d '{"desiredCapabilities":{"browserName": "firefox"}}'`
+1. Repeat!
+
+### Run test
+1. Try create session, such as `curl -X POST http://127.0.0.1:4444/wd/hub/session -d '{"desiredCapabilities":{"browserName": "firefox"}}'`
+1. If you see something similar with `{"state":null,"sessionId":"515be56a...` all right! If not, submit [issue](https://github.com/qa-dev/jsonwire-grid/issues/new)
+
+
+## Configuration
+Configurations are stored in json files. Example:
+```
+{
+  "logger": {
+    "level": "debug"
+  },
+  "db": {
+    "implementation": "local"
+  },
+  "grid": {
+    "client_type": "selenium",
+    "port": 4444,
+    "strategy_list": [
+      {
+        "type": "persistent"
+      }
+    ],
+    "busy_node_duration": "15m",
+    "reserved_node_duration": "5m"
+  }
+}
+```
+
+### Logger - Configuration of logger.
+| Option        | Possible values                     | Description            | 
+| ------------- | ----------------------------------- | ---------------------- |
+| logger.level  | `debug`, `info`, `warning`, `error` | Logging level.         |
+
+### DB - Configuration of storage.
+| Option            | Possible values  | Description                                |
+| ----------------- | ---------------  | ------------------------------------------ |
+| db.implementation | `mysql`, `local` | Select your favorite db, or local storage. |
+| db.connection     | see next table   | DSN for your db.                           |
+
+>Note: Note: Local (in-memory) storage does not support common session storage between grid-instances.
+
+| DB implementation | DSN format                                                                                                                                                              |
+| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| mysql             | [spec](https://github.com/go-sql-driver/mysql#dsn-data-source-name), example `db_user:db_pass@(db_host:3306)/db_name?parseTime=true` (parseTime=true - required option) |
+| local             | omit this property, because every instance have its own in-memory storage                                                                                               |
+
+### Statsd - Configuration of metrics.
+| Option          | Possible values | Description            |
+| --------------- | --------------- | ---------------------- |
+| statsd.host     | `string`        | Host of statsd server. |
+| statsd.port     | `int`           | Port of statsd server. |
+| statsd.protocol | `string`        | Network protocol.      |
+| statsd.prefix   | `string`        | Prefix of metrics tag. |
+| statsd.enable   | `bool`          | Enable metric.         |
+
+### Grid - Configuration of app.
+| Option                      | Possible values          | Description                                                                                                                                                    |
+| --------------------------- | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| grid.client_type            | `selenium`, `wda`        | Type of used nodes.                                                                                                                                            |
+| grid.port                   | `int`                    | Grid will run on this port.                                                                                                                                    |
+| grid.busy_node_duration     | `string` as `12m`, `60s` | Max session lifetime, when timeout was elapsed grid will kill the session.                                                                                     |
+| grid.reserved_node_duration | `string` as `12m`, `60s` | Max timeout between send request `POST /session` and opening the browser window. (Deprecated will renamed)                                                     |
+| grid.strategy_list          | `array`                  | List of strategies, if grid not able create session on first strategy it go to next, until list ends. [Read more about strategies.](#element-of-strategy-list) |
+
+> * `selenium` - [http://www.seleniumhq.org/]()
+> * `wda` - [agent](https://github.com/qa-dev/WebDriverAgent) for [WDA](https://github.com/qa-dev/WebDriverAgent)
+
+### Element of strategy list
+| Option                         | Possible values                                                 | Description                                                                  |
+| ------------------------------ | --------------------------------------------------------------- | ---------------------------------------------------------------------------- |
+| type                           | `string`, see [type of strategy.](#types-of-strategy)           | Host of statsd server.                                                       |
+| limit                          | `int`, unlimited if equals `0`                                  | Max count of active nodes on this strategy.                                  |
+| params                         | `object`, dependent on [strategy type](#types-of-strategy)      | Object describes available nodes, ex. docker config, kubernetes config, etc. |
+| node_list                      | `array`, dependent on [strategy type](#types-of-strategy)       | Array of objects describing available nodes.                                 |
+| node_list.[].params            | `object`, dependent on [strategy type](#types-of-strategy)      | Object of describing node, ex. image_name, etc.                              |
+| node_list.[].capabilities_list | `array`, ex. [{"foo: "bar"}, {"foo: "baz", "ololo": "trololo"}] | array of objects describes available capabilities .                          |       
+
+### Types of strategy
+##### `persistent` - using externally started nodes, same as original selenium grid.
+| Strategy option | Possible values | Description                                          |
+|---------------- | --------------- | ---------------------------------------------------- |
+| limit           | -               | Omit this property, сount of nodes always unlimited. |
+| params          | -               | Omit this property.                                  |
+| node_list       | -               | Omit this property.                                  |
+
+##### `kubernetes` - on-demand nodes in kubernetes cluster.
+| Strategy option           | Possible values | Description                 |
+|-------------------------- | --------------- | --------------------------- |
+| params                    | -               | Omit this property.         |
+| node_list.[].params.image | -               | Docker image with selenium. |
+| node_list.[].params.port  | -               | Port of selenium.           |

config-local-sample.json

@@ -0,0 +1,19 @@
+{
+  "logger": {
+    "level": "debug"
+  },
+  "db": {
+    "implementation": "local"
+  },
+  "grid": {
+    "client_type": "selenium",
+    "port": 4444,
+    "strategy_list": [
+      {
+        "type": "persistent"
+      }
+    ],
+    "busy_node_duration": "15m",
+    "reserved_node_duration": "5m"
+  }
+}

config-sample.json

@@ -1,6 +1,6 @@
 {
   "logger": {
-    "level": 6
+    "level": "debug"
   },
   "db": {
       "implementation": "mysql",

config-test.json

@@ -1,6 +1,6 @@
 {
   "logger": {
-    "level": 3
+    "level": "warning"
   },
   "db": {
       "implementation": "mysql",

config/config.go

@@ -3,7 +3,6 @@ package config
 import (
 	"encoding/json"
 	"errors"
-	"github.com/Sirupsen/logrus"
 	log "github.com/Sirupsen/logrus"
 	"os"
 )
@@ -20,6 +19,7 @@ type Grid struct {
 	Port             int        `json:"port"`
 	StrategyList     []Strategy `json:"strategy_list"`
 	BusyNodeDuration string     `json:"busy_node_duration"`     // duration string format ex. 12m, see time.ParseDuration()
+	// todo: выпилить и сделать равным дедлайну http запроса
 	ReservedDuration string     `json:"reserved_node_duration"` // duration string format ex. 12m, see time.ParseDuration()
 }
 
@@ -36,7 +36,7 @@ type Node struct {
 }
 
 type Logger struct {
-	Level logrus.Level `json:"level"`
+	Level string `json:"level"`
 }
 
 type DB struct {

goreleaser.yml

@@ -4,4 +4,8 @@ build:
     - darwin
     - linux
 hooks:
-    pre: make prepare
+  pre: make prepare
+archive:
+  files:
+    - config-local-sample.json
+    - README.md

logger/logger.go

@@ -4,8 +4,15 @@ import (
 	"github.com/Sirupsen/logrus"
 
 	"github.com/qa-dev/jsonwire-grid/config"
+	"fmt"
 )
 
-func Init(logger config.Logger) {
-	logrus.SetLevel(logger.Level)
+func Init(logger config.Logger) error {
+	level, err := logrus.ParseLevel(logger.Level)
+	if err != nil {
+		return fmt.Errorf("Parse log level, %v", err)
+	}
+	logrus.Infof("Set log level to: %v", level)
+	logrus.SetLevel(level)
+	return nil
 }

main.go

@@ -27,7 +27,10 @@ func main() {
 	if err != nil {
 		log.Fatalf("Problem in loading config from file, %s", err)
 	}
-	logger.Init(cfg.Logger)
+	err = logger.Init(cfg.Logger)
+	if err != nil {
+		log.Fatalf("Problem in init logger, %s", err)
+	}
 
 	statsdClient, err := metrics.NewStatsd(
 		cfg.Statsd.Host,

pool/pool.go

@@ -10,6 +10,7 @@ import (
 )
 
 const (
+	// todo: сейчас не используются, совсем удалить или сделать дефолтные параметры для конфига.
 	defaultBusyNodeDuration     = time.Minute * 30
 	defaultReservedNodeDuration = time.Minute * 5
 )

storage/mysql/factory.go

@@ -21,6 +21,12 @@ func (f *Factory) Create(config config.Config) (pool.StorageInterface, error) {
 		return nil, err
 	}
 
+	err = db.Ping()
+	if err != nil {
+		err = errors.New("Database connection not establish: " + err.Error())
+		return nil, err
+	}
+
 	db.SetMaxIdleConns(0)  // this is the root problem! set it to 0 to remove all idle connections
 	db.SetMaxOpenConns(10) // or whatever is appropriate for your setup.