Gorgonina hyperparameter optimization

[c-bata]

Add an example to optimize hyperparameters for Gorgonia Iris example.

• https://github.com/gorgonia/gorgonia/tree/master/examples/iris
• https://gorgonia.org/tutorials/iris/

Result

(venv) $ go build -o /Users/a14737/go/src/github.com/c-bata/goptuna/bin/gorgonia_iris /Users/a14737/go/src/github.com/c-bata/goptuna/_examples/gorgonia_iris/main.go
(venv) $ ./bin/gorgonia_iris -dataset ./_examples/gorgonia_iris/iris.csv 
2020/04/20 01:47:01 [INFO] Trial finished: trialID=1 state=Complete evaluation=0.000000
2020/04/20 01:47:01 [INFO] Trial finished: trialID=2 state=Complete evaluation=0.813333
2020/04/20 01:47:02 [INFO] Trial finished: trialID=3 state=Complete evaluation=0.960000
2020/04/20 01:47:02 [INFO] Trial finished: trialID=4 state=Complete evaluation=0.626667
2020/04/20 01:47:03 [INFO] Trial finished: trialID=5 state=Complete evaluation=0.600000
2020/04/20 01:47:03 [INFO] Trial finished: trialID=6 state=Complete evaluation=0.880000
2020/04/20 01:47:04 [INFO] Trial finished: trialID=7 state=Complete evaluation=0.833333
2020/04/20 01:47:05 [INFO] Trial finished: trialID=8 state=Complete evaluation=0.980000
2020/04/20 01:47:05 [INFO] Trial finished: trialID=9 state=Complete evaluation=0.000000
2020/04/20 01:47:06 [INFO] Trial finished: trialID=10 state=Complete evaluation=0.826667
2020/04/20 01:47:06 [INFO] Trial finished: trialID=11 state=Complete evaluation=0.953333
2020/04/20 01:47:07 [INFO] Trial finished: trialID=12 state=Complete evaluation=0.873333
2020/04/20 01:47:07 [INFO] Trial finished: trialID=13 state=Complete evaluation=0.953333
2020/04/20 01:47:08 [INFO] Trial finished: trialID=14 state=Complete evaluation=0.953333
2020/04/20 01:47:09 [INFO] Trial finished: trialID=15 state=Complete evaluation=0.833333
2020/04/20 01:47:09 [INFO] Trial finished: trialID=16 state=Complete evaluation=0.906667
2020/04/20 01:47:10 [INFO] Trial finished: trialID=17 state=Complete evaluation=0.960000
2020/04/20 01:47:10 [INFO] Trial finished: trialID=18 state=Complete evaluation=0.693333
2020/04/20 01:47:11 [INFO] Trial finished: trialID=19 state=Complete evaluation=0.840000
2020/04/20 01:47:11 [INFO] Trial finished: trialID=20 state=Complete evaluation=0.966667
2020/04/20 01:47:12 [INFO] Trial finished: trialID=21 state=Complete evaluation=0.966667
2020/04/20 01:47:13 [INFO] Trial finished: trialID=22 state=Complete evaluation=0.960000
2020/04/20 01:47:13 [INFO] Trial finished: trialID=23 state=Complete evaluation=0.000000
2020/04/20 01:47:14 [INFO] Trial finished: trialID=24 state=Complete evaluation=0.973333
2020/04/20 01:47:14 [INFO] Trial finished: trialID=25 state=Complete evaluation=0.840000
2020/04/20 01:47:15 [INFO] Trial finished: trialID=26 state=Complete evaluation=0.953333
2020/04/20 01:47:15 [INFO] Trial finished: trialID=27 state=Complete evaluation=0.966667
2020/04/20 01:47:16 [INFO] Trial finished: trialID=28 state=Complete evaluation=0.960000
2020/04/20 01:47:17 [INFO] Trial finished: trialID=29 state=Complete evaluation=0.920000
2020/04/20 01:47:17 [INFO] Trial finished: trialID=30 state=Complete evaluation=0.960000
2020/04/20 01:47:18 [INFO] Trial finished: trialID=31 state=Complete evaluation=0.960000
2020/04/20 01:47:18 [INFO] Trial finished: trialID=32 state=Complete evaluation=0.000000
2020/04/20 01:47:19 [INFO] Trial finished: trialID=33 state=Complete evaluation=0.966667
2020/04/20 01:47:19 [INFO] Trial finished: trialID=34 state=Complete evaluation=0.953333
2020/04/20 01:47:20 [INFO] Trial finished: trialID=35 state=Complete evaluation=0.973333
2020/04/20 01:47:21 [INFO] Trial finished: trialID=36 state=Complete evaluation=0.000000
2020/04/20 01:47:21 [INFO] Trial finished: trialID=37 state=Complete evaluation=0.973333
2020/04/20 01:47:22 [INFO] Trial finished: trialID=38 state=Complete evaluation=0.000000
2020/04/20 01:47:22 [INFO] Trial finished: trialID=39 state=Complete evaluation=0.833333
2020/04/20 01:47:23 [INFO] Trial finished: trialID=40 state=Complete evaluation=0.966667
2020/04/20 01:47:24 [INFO] Trial finished: trialID=41 state=Complete evaluation=0.960000
2020/04/20 01:47:24 [INFO] Trial finished: trialID=42 state=Complete evaluation=0.966667
2020/04/20 01:47:25 [INFO] Trial finished: trialID=43 state=Complete evaluation=0.000000
2020/04/20 01:47:25 [INFO] Trial finished: trialID=44 state=Complete evaluation=0.960000
2020/04/20 01:47:26 [INFO] Trial finished: trialID=45 state=Complete evaluation=0.973333
2020/04/20 01:47:26 [INFO] Trial finished: trialID=46 state=Complete evaluation=0.000000
2020/04/20 01:47:27 [INFO] Trial finished: trialID=47 state=Complete evaluation=0.960000
2020/04/20 01:47:28 [INFO] Trial finished: trialID=48 state=Complete evaluation=0.966667
2020/04/20 01:47:28 [INFO] Trial finished: trialID=49 state=Complete evaluation=0.000000
2020/04/20 01:47:29 [INFO] Trial finished: trialID=50 state=Complete evaluation=0.960000
2020/04/20 01:47:29 Best evaluation=0.980000 (learning_rate=0.004181)

[c-bata]

Optimize learning rate.

[c-bata]

Following suggestion seems to be more interesting as an Goptuna example.

	var solver gorgonia.Solver
	solverName, _ := trial.SuggestCategorical("solver", []string{"Adam", "Vanilla"})
	if solverName == "Adam" {
		solver = gorgonia.NewAdamSolver()
	} else if solverName == "Vanilla" {
		learnRate, _ := trial.SuggestLogFloat("vanilla_learning_rate", 1e-5, 1e-1)
		solver = gorgonia.NewVanillaSolver(gorgonia.WithLearnRate(learnRate))
	}

I created a pull request: #119

[c-bata]

Maximize accuracy.

[c-bata]

Optimize parameters using TPE.