CakePHP 3 … the app is mature (step 6 — Controller, JSON testing and a little more…)

Ah, after a few lengthy days of work, we have arrived at the conclusion of our development efforts.

In the last post I have done a little recap of the things accomplished so far.

And here, as promised, I am going to piece everything together.
What we haven’t seen so far is the Controller of the application, which is responsible for taking the requests from jQuery and passing them on to your Model layer.

One thing you’ve probably noticed is that I have not talked about any “views” in our application. (Other than index.ctp, which we’ve prepped back here).

For this application we don’t need to create any view files like we typically would, if our response was a traditional HTML page to the user. In this case, the Controller only responds as JSON, and for that purpose CakePHP 3 provides a special View object for us.

If you recall, we had some interesting “request” URL’s to our server (i.e. /todos/get.json). The .json extension tells our controller that we are requesting a JSON response (vs the default HTML, for example). To ensure that our application “understands” the meaning of this extension, it is important to enable  Router::extensions(['json']); in config/routes.php.

Now, let’s take a look at the src/Controller/TodosController.php

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
<?php
namespace App\Controller;

use Cake\Core\Configure;
use Cake\ORM\TableRegistry;

class TodosController extends AppController {

/**
 * initialize method
 *
 * @return void
 */

  public function initialize() {
    parent::initialize();
    $this->loadComponent('RequestHandler');
  }

/**
 * main action for the application
 *
 * @return void
 */

  public function index() {
    //this method is intentionally left blank
  }

/**
 * add() action to create a new to-do
 *
 * @return void
 */

  public function add() {
    $response = ['result' => 'fail'];
    $errors = $this->Todos->validator()->errors($this->request->data);
    if (empty($errors)) {
      $todo = $this->Todos->newEntity($this->request->data);
      if ($this->Todos->save($todo)) {
        $response = ['result' => 'success'];
      }
    } else {
      $response['error'] = $errors;
    }
    $this->set(compact('response'));
    $this->set('_serialize', ['response']);
  }

/**
 * gets either done or incomplete to-do's depending on the status
 *
 * @param int $status 0/1 incomplete/complete
 * @return void
 */

  public function get($status = 0) {
    $todos = $this->Todos->find('recent', ['status' => $status]);
    $this->set(compact('todos'));
    $this->set('_serialize', ['todos']);
  }

/**
 * marks the to-do as complete, i.e. changes is_done to 1
 *
 * @param int $todoId id of the record to mark as done
 * @return void
 */

  public function finish($todoId = null) {
    $response = ['result' => 'fail'];
    if (!is_null($todoId)) {
      $todos = TableRegistry::get('Todos');
      $todo = $todos->get($todoId);
      $todos->patchEntity($todo, ['is_done' => 1]);
      if ($todos->save($todo)) {
        $response = ['result' => 'success'];
      }
    }
    $this->set(compact('response'));
    $this->set('_serialize', ['response']);
  }
}

Most of the magic begins with the RequestHandler component. By adding it to the application in the initialize() method, we make our Controller much more powerful because now it can accept AJAX requests and easily respond with JSON.
You can quickly examine add(), get() and finish() actions to spot $this->set('_serialize', $something);, which essentially creates a JSON (or JSONP) response from a view variable populated by $this->set().

Specifically if you look at the get() action, you’ll probably recall our test for findRecent() in the previous post. Here you can see how we can call this method from the controller:

$todos = $this->Todos->find('recent', ['status' => $status]);

Additionally, because we’ve already took care of data sanitization and formatting in our “Todo” Model, we can pretty safely return the data back to the browser.
Eventually, depending on what the server sends back to the front-end our jQuery script will “decide” the outcome of how the the UI is affected.
In one case we’ll show an error message, in another we will update the list of the complete tasks or perhaps mark a task as complete and make it disappear from your “to-do” list and appear on your “recently done” list.
To get a little glimpse of that, you can look in the add() action.
There, for example, I am getting the validation errors from the Model by using $this->Todos->validator()->errors($this->request->data); If the validation fails our “to-do” cannot be saved, and the error is returned to the user (lines 44, 45); otherwise we proceed with the save() and return “success”.

As I did for the Table object, it is important to provide unit tests for the Controller:

srs/tests/TestCase/Controller/TodosControllerTest.php

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
<?php
namespace App\Test\TestCase\Controller;

use Cake\I18n\Time;
use Cake\Routing\Router;
use Cake\TestSuite\IntegrationTestCase;

class TotosControllerTest extends IntegrationTestCase {

/**
 * fixtures
 *
 * @var Fixture
 */

  public $fixtures = ['app.todos'];

/**
 * test add() method
 *
 * @return void
 */

  public function testAdd() {
    $this->configRequest([
      'headers' => ['Accept' => 'application/json']
    ]);

    $this->post(Router::url(
      ['controller' => 'todos',
        'action' => 'add',
        '_ext' => 'json'
      ]),
    ['todo' => 'run test']);

    $this->assertResponseOk();
    $expected = [
      'response' => ['result' => 'success'],
    ];
    $expected = json_encode($expected, JSON_PRETTY_PRINT);
    $this->assertEquals($expected, $this->_response->body());
  }

/**
 * test get() method
 *
 * @return void
 */

  public function testGet() {
    $this->configRequest([
      'headers' => [
        'Accept' => 'application/json'
      ]
    ]);

    $this->post(Router::url(
      ['controller' => 'todos',
        'action' => 'get',
        '_ext' => 'json'
      ])
    );
    $this->assertResponseOk();

    $expected = [
      'todos' =>
        [
          [
            'id' => 1,
            'todo' => 'First To-do',
            'created' => 'on 11/21/13',
            'updated' => 'on 11/21/13',
            'is_done' => false
          ]
        ],
    ];
    $expected = json_encode($expected, JSON_PRETTY_PRINT);
    $this->assertEquals($expected, $this->_response->body());

    // get completed to-do's
    $this->post(Router::url(
      ['controller' => 'todos',
        'action' => 'get',
        '_ext' => 'json',
        1
      ])
    );
    $this->assertResponseOk();

    $expected = [
      'todos' => [
        [
          'id' => 2,
          'todo' => 'Complete To-do',
          'created' => 'on 11/21/13',
          'updated' => 'on 11/21/13',
          'is_done' => true

        ]
      ]
    ];
    $expected = json_encode($expected, JSON_PRETTY_PRINT);
    $this->assertEquals($expected, $this->_response->body());
  }

/**
 * test fnish() method
 *
 * @return void
 */

  public function testFinish() {
    $this->configRequest([
      'headers' => [
        'Accept' => 'application/json'
      ]
    ]);

    $this->get(Router::url(
      ['controller' => 'todos',
        'action' => 'finish',
        '_ext' => 'json',
        2
      ])
    );
    $this->assertResponseOk();
    $expected = [
      'response' => ['result' => 'success'],
    ];
    $expected = json_encode($expected, JSON_PRETTY_PRINT);
    $this->assertEquals($expected, $this->_response->body());

    // get completed to-do's
    $this->post(Router::url(
      ['controller' => 'todos',
        'action' => 'get',
        '_ext' => 'json',
        1
      ])
    );
    $this->assertResponseOk();
    $this->assertResponseContains('"is_done": true', $this->_response->body());
  }

}

The test builds on top of what we’ve done in the last post. Although I now know that the “get” and “save” operations are working well from the data (or Model) layer perspective, Controller testing helps to ensure that front-end response is also up to par.
As such, I can test the validity of the response when certain action is requested, the format (i.e. JSON) and finally, by using the fixture data, I can assert some expectations of the actual returned data.
I won’t go into the details of the code here, because testing is described very well in the CakePHP cookbook, and the code, IMO, is relatively easy to read.

Now we have all the pieces put together:

  • UI including CSS and mark-up
  • jQuery for the front-end and user experience logic as well as dispatching of events
  • Model layer in CakePHP to handle database interaction
  • And finally the controller to dispatch requests and responses

One last thing before we are ready to launch the app…
If you currently went to the URL root of your application at http://local.todo/ (for example), you would see a default CakePHP homepage rather than the application. The simple way to ensure that the user is always redirected to the proper destination, which would actually be http://todo.local/todos/index/ (or just /todos/), is to edit our routes file.

config/routes.php

$routes->connect('/', ['controller' => 'Todos', 'action' => 'index']);

All we did here is replaced the default (root) “/” route with the new destination.

Well done for following along. To grab the code and to help to improve this app further, please check out the github repo.

CakePHP 3 … fully grown (step 5 — Model layer and testing)

Finally we are the point where some interesting things are about to happen.

Let’s recap a little so far:

So here comes the fun part.

Let’s build the server-side part of our application in CakePHP 3.

Hmm… Where to start?

The Model layer.

I have to say that if you are familiar with CakePHP 2.x ORM, working with the new Model layer is either going to bring you a great deal of pain or pleasure… (or both in that order).
Indeed, there will be a bit of a learning curve to get used to the new and improved way of doing things, so although I am a bit sorry to say it, but if you’ve spent the last two years learning the intricacies of CakePHP 2.x ORM you are in for a bit of a surprise… although don’t despair, your knowledge will not go wasted.

In CakePHP 3 the ORM has been rebuilt from the ground-up and personally I find it a lot more flexible, better structured and “officially” speaking the separation of concerns principle is truly respected.

Back in the day we used to have a CakePHP Model and that was pretty much all there is to it (the Model layer). You could call a find() method (or some variation thereof) to retrieve your data and you’d get an array of data back.
For most purposes it worked very well, but as soon as you needed to do something more complicated (sub-queries anyone?)… things got rather nasty and in turn you could see plenty of poorly written CakePHP apps.

But let’s not dwell on the past, as today we have:

  • Table object, which provides a direct representation of your table.
  • Entity object, basically represents a record row (i.e. if you have a bunch of Products in a table named “products”, then a single product or item representation would be handled by the Entity object named “Product”, in this example.)… so to keep in mind: Table — collection of records, Entity — single row/record.
  • Query object, allows for construction and manipulation of queries. A little more on that below.

All of these are described at length in the CakePHP Cookbook, therefore I don’t think there is a need to go over each one in detail.

Instead, let’s just take a look at the code for our table, if you recall, we have table called “todos”, therefore a file representing this table should be created ( src/Model/Table/TodoTable.php):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
<?php
namespace App\Model\Table;

use Cake\I18n\Time;
use Cake\ORM\Query;
use Cake\ORM\Table;
use Cake\Validation\Validator;

class TodosTable extends Table {

/**
 * initialize method
 *
 * @param array $config list of config options
 * @return void
 */

  public function initialize(array $config) {
    $this->addBehavior('Timestamp', [
      'events' => [
        'Model.beforeSave' => [
        'created' => 'new',
        'updated' => 'always'
      ]
    ]]);
  }

/**
 * Default validator method
 *
 * @param Validator $validator cakephp validator object
 * @return Validator $validator cakephp validator object
 */

  public function validationDefault(Validator $validator) {
    $validator
    ->allowEmpty('todo', 'update')
    ->notEmpty('todo');

    return $validator;
  }

/**
 * Custom finder method, returns recent to-do's based on status
 *
 * @param Query $query  cakephp query object
 * @param array $options list of options
 * @return query $query cakephp query object
 */

  public function findRecent(Query $query, array $options = ['status' => 0]) {
    return $query
        ->where(['is_done' => $options['status']])
        ->order(['updated' => 'DESC'])
        ->formatResults(function ($results, $query) {
          return $results->map(function ($row) {
            $timeCreated = new Time($row->created);
            $timeUpdated = new Time($row->updated);

            $row->created = $timeCreated->timeAgoInWords();
            $row->updated = $timeUpdated->timeAgoInWords();
            $row->todo = htmlspecialchars($row->todo);

            return $row;
          });
        });
  }
}

Although it looks like a lot of code, most of our application is contained right here. Let’s try to break it down.

First, I am attaching the Timestamp behavior to handle our “created” an “updated” fields. (Note I diverged slightly from the book where the default field is “modified”, and not “updated”… however the problem is readily fixed by providing a few extra settings as you see on lines 20 -22).

Next, comes our validationDefault() method. Although it looks a bit different from the way things were in the previous versions of CakePHP, the code within it should be pretty straight forward. I am checking to make sure that the “todo” field is not empty. During the “update” of the record, however, there is no need to attempt to validate this field.

The next method findRecent() is a custom finder method, which is quite powerful in the new ORM.
Notice, that this method does not return a result set. Traditionally you’d want this method to return a query object. What happens here is that the query object gets created and the actual SQL is ready to be executed at this point. However to trigger the execution of the SQL to actually “extract” your results requires another step (more about this when I talk about controllers).

There is a couple of other interesting things here…
I am using both formatResults() method and map() methods (thanks, Jose, for the hint) to modify certain things in my result-set. To understand the code a little better what we do here is modify each row value to our liking. For example I am protecting/sanitizing the “todo” field by using PHP’s htmlspcialchars() method. Additionally I am modifying the timestamp to be human-readable rather than Database date-time format, which looks rather ugly.

That’s all there is to our Table. This single method will allow us to get both finished and incomplete to-do’s by toggling the ‘status’ option.

Next, I must talk about unit testing. Actually as a somewhat decent developer I should start every application the TDD (test driven development) way. What that means I write my tests before I ever think about writing the production code.

For the sake of this example, please forgive me for not doing so from the get-go, but here comes our first test. After all, if our table has been setup the right way then the testing should be a no-brainer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
<?php
namespace App\Test\TestCase\Model\Table;

use Cake\I18n\Time;
use Cake\ORM\Query;
use Cake\ORM\TableRegistry;
use Cake\TestSuite\TestCase;

class TodosTest extends TestCase {

/**
 * fixtures
 *
 * @var array
 */

  public $fixtures = ['app.todos'];

/**
 * setUp() method
 *
 * @return void
 */

  public function setUp() {
    parent::setUp();
    $this->Todos = TableRegistry::get('Todos');
  }

/**
 * test saving of a to-do, validation
 *
 * @return void
 */

  public function testSaving() {
    $data = ['todo' => ''];
    $todo = $this->Todos->newEntity($data);
    $resultingError = $this->Todos->validator()->errors($data);
    $expectedError = [
      'todo' => [
        'This field cannot be left empty'
      ]
    ];
    $this->assertEquals($expectedError, $resultingError);

    $total = $this->Todos->find()->count();
    $this->assertEquals(2, $total);

    $data = ['todo' => 'testing'];
    $todo = $this->Todos->newEntity($data);
    $this->Todos->save($todo);
    $newTotal = $this->Todos->find()->count();
    $this->assertEquals(3, $newTotal);
  }

/**
 * test custom finder method
 *
 * @return void
 */

  public function testRecent() {
    $result = $this->Todos->find('recent', ['status' => 0]);
    $recent = $result->first()->toArray();
    $expected = [
        'id' => 1,
        'todo' => 'First To-do',
        'created' => 'on 11/21/13',
        'updated' => 'on 11/21/13',
        'is_done' => false
    ];

    $this->assertEquals($expected, $recent);
  }

/**
 * test saving of a to-do with evil data
 *
 * @return void
 */

  public function testSaveEvilScript() {
    $data = ['todo' => '<script>alert("hi")</script>', 'is_done' => 1];
    $todo = $this->Todos->newEntity($data);
    $this->Todos->save($todo);
    $newTotal = $this->Todos->find()->count();
    $this->assertEquals(3, $newTotal);

    $result = $this->Todos->find('recent', ['status' => 1])->where(['id' => 3])->first();
    $this->assertEquals('&lt;script&gt;alert(&quot;hi&quot;)&lt;/script&gt;', $result->todo);
  }

/**
 * test to make sure custom finder returns the dates in a human-readable format
 *
 * @return void
 */

  public function testFindTimeAgoInWords() {
    $todos = TableRegistry::get('Todos');
    $todo = $todos->get(1);
    $todos->patchEntity($todo, ['updated' => new Time(date('Y-m-d H:i:s', strtotime('-3 seconds ago')))]);
    $todos->save($todo);
    $result = $todos->find('recent', ['status' => 0])->where(['id' => 1])->first();
    $this->assertContains('second', $result->updated);

    $todos = TableRegistry::get('Todos');
    $todo = $todos->get(1);
    $todos->patchEntity($todo, ['created' => new Time(date('Y-m-d H:i:s', strtotime('-3 seconds ago')))]);
    $todos->save($todo);
    $result = $todos->find('recent', ['status' => 0])->where(['id' => 1])->first();
    $this->assertContains('second', $result->created);
  }
}

Although this test is lengthy in code, it is rather simple in what it actually accomplishes.

First we try testSaving()… this ensures that both the validation and the actual storage of the data to the database, works as expected.
(Based on the assertions you can see that we expect things to error-out with an empty “to-do” and save correctly, when a “to-do” has some value, on lines 42 and 51 respectively).

Additionally I test that an “evil” script cannot be inserted into our page by ensuring that the htmlspecialchars() method works its magic on a “to-do”. Take a look at line 59 in the table and the above method testSaveEvilScript().

Lastly, I am testing testFindTimeAgoInWords(), which ensures that a human-readable timestamp is returned, rather than something that looks like a posting on a personal ad for robots.

Of course, our testing would not be possible without a fixture (to better phrase it, having some fixture/seed data to work with, makes testing a lot easier):

public $fixtures = ['app.todos'];

I will provide you the sample fixture below.

For now suffice it to say, that I am done with the model layer and its testing. Although we have not looked at the entity object in detail, it is simply not required for our application… and CakePHP is smart enough to instantiate entity objects for you and hydrate them as necessary. More on that, in the CakePHP cookbook.

Finally, I will talk about the controller (and testing of it, of course, in the following post).

… and as promised here’s a sample fixture, which I am using for my tests:

<?php
namespace App\Test\Fixture;

use Cake\TestSuite\Fixture\TestFixture;

class TodosFixture extends TestFixture {

  public $import = ['table' => 'todos'];

  public $records = [
    [
      'id' => 1,
      'todo' => 'First To-do',
      'is_done' => '0',
      'created' => '2013-11-21 12:00:00',
      'updated' => '2013-11-21 12:00:00'
    ],
    [
      'id' => 2,
      'todo' => 'Complete To-do',
      'is_done' => '1',
      'created' => '2013-11-21 12:00:00',
      'updated' => '2013-11-21 12:00:00'
    ]
  ];
}

CakePHP 3 … growing up (step 4 — it’s AJAX time)

Taking a look back at where I have left off, it’s time to start working on the interactive part of our application. It should come as no surprise that most of the actions taken in the app will be AJAX-based. In other words, submitting a form or clicking a checkbox will be handled by jQuery and in turn will trigger something on the server.

You can imagine that when submitting the form, I will be “adding” a to-do to the database, therefore calling:

TodosController::add()

Likewise, when I finish the task, by clicking on the checkbox, the following call should be made:

TodosController::finish($todoId)

All of this is handled by “attaching” jQuery events to the elements such as form and checkbox. This allows me to fire off ajax requests as necessary and trigger various actions in the controller.

If you recall back in part 3 we’ve included app.js file in our application.

Let’s take a look at it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
var TodoApp = {};

(function(){
  TodoApp.getTodos = function() { $.get('/todos/get.json', function(response) {
    $label = $('#incomplete-label');
    $incompleteDiv = $('#incomplete-to-dos');
    $incompleteDiv.empty();
    if (response.todos.length === 0) {
      $label.hide();
      $incompleteDiv.append('<div class="incomplete-todo">All done. Have a nice day (or add a new to-do above).</div>');
    } else {
      $label.show();
      $.each(response.todos, function(key, value) {
        $incompleteDiv.append('<div class="incomplete-todo" id="incomplete-' + value.id +'"><label for="todo_' + value.id + '">' + value.todo + ' <input id="todo_' + value.id + '" class="todo-checked" type="checkbox" /></label><div class="small-done">' + value.created + '</div></div>');
        $incompleteDiv.show('highlight');
      });
    }
  });
};

  TodoApp.getDone = function() { $.get('/todos/get/1.json', function(response) {
      $doneDiv = $('#done');
      $doneDiv.empty();
      $.each(response.todos, function(key, value) {
        $doneDiv.append('<div class="finished-task"><div class="finshed-task-text">' + value.todo + '</div><div class="small">' + value.updated + '</div></div>');
      });
    });
  };

  TodoApp.finishTask = function(id) {
    $.get('/todos/finish/' + id + '.json',
      function(response) {
        if (response.response.result == 'success') {
          $('#incomplete-' + id).hide('explode');
          $('#incomplete-' + id).remove();
          TodoApp.getTodos();
          TodoApp.getDone();
        } else if (response.response.result == 'fail') {
          console.log('fail');
        }
      }
    );
  };

})();

(function($) {
  $("#add-to-do").submit(function(event) {
    $('#todo-error').remove();
    $('.form-group').removeClass('has-error');
    var $form = $(this),
      todo = $form.find("input[name='to-do']").val(),
      url = $form.attr('action');

    var posting = $.post( url, { todo : todo } );
    posting.done(function( response ) {
      if (response.response.result == 'success') {
        $('#incomplete-to-dos').empty();
        $('#inputLarge').val('');
        TodoApp.getTodos();
      } else if (response.response.result == 'fail') {
        $('.form-group').addClass('has-error');
        $('#task-input').append('<div class="error" id="todo-error">' + response.response.error.todo + '</div>');
      }
    });
    event.preventDefault();
  });

  $(document).on('click', ':checkbox', function() {
    var id = $(this).attr('id').replace('todo_', '');
    TodoApp.finishTask(id);
  });

  TodoApp.getTodos();
  TodoApp.getDone();
})(jQuery);

There’s a little too much code here to go over everything, but I will point out the places of interaction between our front-end (jQuery) and back-end (CakePHP 3).

First, there are a couple of functions that I need to have:

getTodos() //line 4
getDone() //line 21
finishTask() //line 30

All of these functions will call our controller actions.

Very importantly, on line 48:

$("#add-to-do").submit()

We submit our form (which is really a single input field) with the name of the “to-do” that is going to be stored in the DB.
Another important thing to point out here (lines 61 – 63), is that I will rely on the response from the server to display the error messages, this allows me to leverage CakePHP’s server-side validation without any effort.

Finally on line 69 we have:

$(document).on('click', ':checkbox' ...

This binds the “click” event to all the checkboxes in the browser window. Because it’s only a one-page app and the only purpose a checkbox is serving is to “check off” or “finish” a to-do, then I can use this is method as a fast way to attach an event to any checkbox. Furthermore, you can see that when the checkbox is actually clicked, the finishTask() function is called (line 71):

TodoApp.finishTask(id);

In turn, this function triggers our controller TodosController::finish($todoId)

Overall, you can think of this file as a dispatcher of sorts.

When ceratain events are triggered (i.e. click, submit, page load) the app.js + some jQuery magic tells our server to kick-off a process, such as data validation, and then, hopefully, storage into the Database. Or, in the case of clicking a checkbox to “finish” a to-do, we send the request to the server via the finishTask() function and eventually update the record in the DB (I’ll show you this part in the following post).

Of course, in order to do all that, we need to build the rest of the application. The server-side of things. The part that interacts with our Database and sends the information (such as validation errors, or a list of tasks which are still incomplete) back to to the browser… and that’s where CakePHP comes in. Stay tuned.

Cake3 … baby steps (step 3 — let’s get to work, front-end preparation)

In part 1 and part 2 of these mini-series, I went over the basics of installation of CakePHP 3, as well as setting up the initial database and table for our application.

Now that the data modeling part is done :), it’s time consider the actual application.

So what am I building?

It is indeed a to-do app, a very basic checklist of things “to do” with the ability to mark them as “done” as well the list of some recently added and completed ones. There’s about a million of such apps out there, so my goal was to keep it simple and to show off a couple of things as far as new features of CakePHP 3 are concerned.

As I kind of hinted in the previous post, the app is only going to work off of a single table, and all-in-all it will be just a one-page app.

For the front end styling and CSS it almost goes without saying, that I am just going to stick with Bootstrap. As a matter of fact, to get some basic colors and elements that are slightly different from the default look & feel of bootstrap, I’ll just grab this particular theme:
http://bootswatch.com/flatly/.

Once, I download the bootstrap.min.css. file and place it in the webroot/css directory, it’s time to go ahead and modify the default layout file.
Template/Layout/default.ctp

<!DOCTYPE html>
<html>
<head>
  <?= $this->Html->charset() ?>
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>
    <?= 'To-do : an awesome CakePHP 3 app : ' . $this->fetch('title') ?>
  </title>

  <?= $this->Html->css('bootstrap.min') ?>
  <?= $this->Html->css('app') ?>

  <?= $this->fetch('meta') ?>
  <?= $this->fetch('css') ?>
</head>
<body>
  <div id="container">
    <?= $this->fetch('content') ?>
  </div>

  <script src="//ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
  <script src="//ajax.googleapis.com/ajax/libs/jqueryui/1.11.2/jquery-ui.min.js"></script>
  <?= $this->fetch('script') ?>
</body>
</html>

This layout is the default “template” that our entire site will use. (Yes, the entire one page). Here we can load all the necessary CSS and JS that will be used throughout our application. As you see we’ve added jQuery, jQuery UI and Bootstrap CSS. Additionally there will be a little CSS file just for our app (called app.css). You can see it loaded at the top of the page. This is where you can add and override some default styling. It’s a cleaner way to make your changes, rather than messing around in the bootstrap.css file directly.

Anyways, I do have one page (or view) to create. Let’s take a look here:
src/Template/Todos/index.ctp

<?php $this->Html->script('app', ['block' => true]); ?>

<div class="col-md-4 col-md-offset-4">
  <form action="/todos/add.json" class="form-inline" role="form" id="add-to-do">
    <div class="form-group">
      <div class="input-append" id="task-input">
        <input class="form-control input-lg" name="to-do" type="text" id="inputLarge" placeholder="Enter a task...">
        <button type="submit" class="btn btn-primary btn-lg">Add To-do</button>
        </div>
      </div>
  </form>
  <div class="task-container" id="to-dos">
    <form action="/todos/finish.json" class="form-inline" role="form" id="finish-to-do">
      <div id="incomplete-label"><h5>To do:</h5></div>
      <div class="form-group" id="incomplete-to-dos"></div>
    </form>
  </div>
  <div class="task-container">
    <div id="done-label"><h5>Recently done...</h5></div>
    <div id="done"></div>
  </div>
</div>

The very first line will load a file from webroot/js/app.js, into our layout (remember $this->fetch(‘script’))? …
I am not making a use of the CakePHP’s Form Helper here, for a couple of reasons:

  • I don’t really have any forms, only one input field and technically just one checkbox…
  • Most events will be captured and processed by jQuery, so the field names aren’t as important. Also error handling is done a little differently, so again no real need to rely on the input() method.
  • It’s much faster (although less secure) to just write simple HTML in a case like this one

If you take a look at the form URL’s that is what I’ll need to build in CakePHP to tie everything together. Based on the structure alone, that hints at the fact that we’ll have some sort of:

Todos::add() and Todos::finish()

controller and action.

With all that said, the basic set up for the front-end is actually done. In the next post I’ll talk about adding some logic to both the front-end and connecting everything to our back-end (via the Controller and the Model layer).

For now, here’s a temporary screenshot of what the application will look like (albeit, still in-progress)…
Also, I hope this makes things a bit more clear as to how everything is going to work, once I build out the rest of the application.

todo cakephp 3 app

Todo app

CakePHP 3 – DebugKit, where are you?

Debug Kit, one of the most awesome plugins for CakePHP, is now installed by default with the cake core.

However, I recently setup a brand new Cake 3 app, and DebugKit was nowhere to be seen in my app.

What gives?

Checking all the usual suspects, like making sure debug is true and the plugin is loaded in app/config/bootstrap.php

Plugin::load('DebugKit', ['bootstrap' => true]);

I resorted to checking out the CakePHP error log file (app/tmp/logs/error.log)… And lookly-look what we have there:

Warning: DebugKit not enabled. You need to either install pdo_sqlite, or define the "debug_kit" connection name.

Well, it just happens that not only my Cake install was fresh, but also my Ubuntu.

For me running:

sudo apt-get install php5-sqlite

.. fixed the problem.

But there are more stubborn cases and the solution for those is described here.

Cake3 … baby steps (step 2 — setting up)

m4s0n501

Now that we have cake installed, and our fresh application is up and running, it’s time to setup some things and prepare for development.

I am going to use good ol’ MySQL for my DB, so first of all I need to create a database called “todo” and provide the right login credentials in
todo/config/app.php
I am going to setup both the “default” and “test” connections, kind of like so:

'Datasources' => [
    'default' => [
      'className' => 'Cake\Database\Connection',
      'driver' => 'Cake\Database\Driver\Mysql',
      'persistent' => false,
      'host' => 'localhost',
      'username' => 'root',
      'password' => 'password',
      'database' => 'todo',
      'encoding' => 'utf8',
      'timezone' => 'UTC',
      'cacheMetadata' => true,

Next, I hear that cake migrations got even more awesome in the new version. Since they are leveraging Phinx, a very solid migrations tool. So I am definitely going to use cakephp migrations to keep my database changes organized.

The installation is now done via composer.
Just add the following line to your todo/composer.json

"cakephp/migrations": "dev-master"

You can place this in the require{} section.

Now, back to the terminal and your main app directory and simply run:

composer update

This should update any dependencies and install cakephp/migrations plugin.
Don’t forget to add Plugin::load(‘Migrations’); to your config/bootstrap.php

You can verify that the plugin is available by running bin/cake from your main application directory.

Let’s get our first migration setup, by running:

bin/cake migrations create Initial

Basically this will create a file for our first migration. My plan is to have a whole of one table for the application.

We can easily define the table and add it to the database using the migrations plugin.
Open up the migrations file in config/Migrations/[something]_initial.php and let’s take a look at the change() method.

This is where the definition of the table should go. Generally speaking change() method replaced up() and down() for the most part, you can still make use of all if you have to get very granular, but when it comes to creating and dropping (when reverting) a table… change() works perfectly well.

Let’s see how to define a table inside the change() method:

public function change()
    {
        // create the table
        $table = $this->table('todos');
        $table->addColumn('todo', 'string', ['limit' => 200])
              ->addColumn('created', 'datetime')
              ->addColumn('updated', 'datetime')
              ->addColumn('is_done', 'boolean')
              ->create();
    }

Notice, that I don’t even have to supply any information about a Primary Key column.
By default the migration will create and “id” column which is a PK and AUTO_INCREMENT’ed.
Sweet.

Give it a try:

bin/cake migrations migrate

If all goes well, you should now have a new table ready in your database.

Cake3 … baby steps (step 1 — getting started)

With the recent announcement of CakePHP 3 (beta 2) being released… it is really tempting to see where we are today.
And oh boy… how things have changed.

First things first, let’s get a hold of our the latest cake core and setup our local environment to start baking.

One thing that jumps out right away is the use of composer to install cake. This is not a requirement, but is certainly a welcome addition.

I just happen to have composer installed already, so once I “cd” into the directory of my choice, I am ready for CakePHP installation.

sudo composer self-update
composer create-project --prefer-dist -s dev cakephp/app todo

So what now?
We should have a new directory called “todo” (that’s just a name of my app). Inside you will notice a new directory called “vendor”, this is where all composer-compliant modules live by default.
Cake core is now located there and so are any other 3rd party libraries that you might use.

All of this goes to say that CakePHP now follows the same standard of autoloading (PSR-4) as many other popular PHP frameworks and libraries.
To some degree standardization, especially in loosely defined PHP world is a good thing.

Cake3 comes with its own web server, and you are welcome to test it at your leisurely time.

I’m going to go ahead and try to setup an apache virtual host and see if that works.

I need an entry in my /etc/hosts for my new “todo” app.

127.0.0.1     todo.local

Should do it…

To setup a virtual host (apache2), let’s cd to /etc/apache2/sites-available

We’ll create a new file called todo.local.conf, with the following contents:

<VirtualHost *:80>
    ServerName todo.local
    ServerAlias todo.local

    DocumentRoot /home/teknoid/work/todo/webroot
    <Directory /home/teknoid/work/todo/webroot>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride All
        Order deny,allow
        allow from all
    </Directory>
</VirtualHost>

Now let’s enable the site and reload the config:

sudo a2ensite todo.local
sudo service apache2 reload

If all goes well, you should have a fresh CakePHP 3 app when you visit http://todo.local/

Now, that we are ready to start baking… we’ll take a look at some additional setup and development tools to get us going in the next post.

I can’t get my estimations right… Top issues and remedies

Oh man, the topic of accurate estimations in SCRUM.

This is probably one of the most common battles between business owners and technology.

I’d like to point out one thing: the estimates are never accurate.
But how do you estimate? Story points? Hours? Both?

It always seems like whenever you work with a new project a SCRUM approach (especially when it comes to estimations) is handled slightly differently. Some teams value estimations and watch the burn-down chart daily. Some just slap on a number and hope for the best.

I’m going to say that based on my personal experience the number one issue with poor estimations is extremely poorly written user stories.

Without much delay I’d like to talk about issues that can help improve many SCRUM implementations, and even if it doesn’t work for your team… maybe an idea will be born from something listed below.

Let’s take a look…

1. Lack of well defined acceptance criteria

Besides getting something done it is also important to make sure that was done (programmed) is what was intended to be delivered and consequently accepted.

Here’s a typical example of a user story:

Summary:
As a user, I can view the report of all orders and filter them by date, name, and other criteria

Description:
User should have the ability to access the report at:
host/admin/orders/
– filter by date (from – to)
– filter by name
– filter by amount (from – to)
– columns should be sortable

Awesome, right?
Wrong.

Even if I am a developer intimately familiar with the system, I would not be able to estimate very accurately.
As a developer I would have two options:
1. Go on a hunt for missing information (let’s see below)
2. Make a bunch of assumptions (can’t spell assumption without ass)..

Let’s make a bunch of assumptions.. shall we?

The report is in the “admin” section, the user should be logged in as admin (let’s hope there’s only one tier of admin users and only that one tier should be logged in.. we don’t know)
I presume that the report should look like the other ones in the system so far, and make my best attempt at the design.

Four days later the report is loading, it looks good, the filters are working the user has to be logged in as admin. Columns are sortable. Not much to add.
Time to demo the story to the stakeholders… after some interesting conversation the story is rejected.

2. A story that requires research

Why did the above story get rejected?
Well it simply didn’t pass the acceptance criteria set by the business owners. Too bad we found out about this later.
The pagination was off… and apparently only shipping managers and super-admins should have had access…
There were a couple of other minor issues… but one of the required columns was missing (and apparently the data we need is stored in a completely arbitrary table elsewhere).

The number of reasons the story was rejected could go on forever.

The real question is how can we anticipate all these changes/addition ahead of time?
Simple answer: involve business early.

More complicated answer would be to introduce “spikes” into your sprints. This becomes a business practice not a “let’s try to have a meeting about this” type of deal.
Spikes are usually viewed as oddities and negative things. Perhaps so.
But introduction of spikes is often necessary to weed out the problems.

The idea of course is that over time you need to rely less on spikes and the user stories become better defined.

Then comes the question of accountability. If during research and analysis you don’t have an agreed upon design, a clear definition of what should be on the report and who should access it… then you’ve done your analysis poorly. And hopefully you would know so ahead of time, because a spike just like a regular user story should be estimated, demonstrated and accepted. Whether it’s a Photoshop mock-up or a working prototype it’s best to get feedback early and often. Spikes help to accomplish exactly that.

As developer it is simply best to find out where your development user story is headed before you get to coding. If the business is happy with the proposed direction, then chances are the outcome will be as desired.

Getting “on the same page” early is good and necessary.

3. We forgot about QA

Yes we did. QA is not there to break things… if developers aren’t working with QA early and often, then something is going to naturally break eventually.
I know how to develop and how to make something work. But I don’t have desire nor time to figure out all the scenarios and permutations required to potentially break something or otherwise invalidate the user story.

Going back to the point of acceptance criteria… As a developer, QA should be your first line of defense against poorly written acceptance criteria. During your pre-planning meeting, review of backlog or at the worst case at the beginning of the sprint as developer it should be a necessity to meet with QA and discuss the story at hand.
I need to know what the QA will be looking for so I can write the code in such a way that it would pass QA acceptance.

With that said, I hope that my research for the user story has been thorough enough and I can actually estimate the task much more accurately.
If I still feel that we have unanswered questions it is best to introduce a new spike or escalate the story ASAP.

Communicating problems in SCRUM is better before the beginning of the work than after.

And remember to include QA time in your estimations. If you have 4 days (approximately) to work on a task, you better not start 4 days before the end of the sprint, or the story simply won’t make it. Consider that QA needs 2 days to test and there’s a day of defect fixing. So the total estimation cannot be done just from the developer’s point of view.

Not only that, a story that might be simple to code could be much more complicated to test and that should be accounted for whether in time, resource or complexity estimations.

4. Technology needs a strong voice

But what happens if all of the above was done right, yet business owners still reject the user story.

Well, then simply put its their own fault. Yet often there isn’t anybody to defend the developer.
If business owners were involved in the process from the start, they simply have no room to say that this is “not what we wanted”.
Granted, anyone could say what they want, and that the previous designs and discussions were just a fabrication and food for thought… At which point I would seriously consider employment elsewhere.

However, that’s not really the solution.

Either during planning or acceptance someone who represents technology (without the fear of being fired) needs to point out where things went off course and how to mitigate the issue going forward. Hopefully those responsible will behave better next time and crisis will be avoided.

But in case of emergency break glass, any glass… and look for a new job.

5. Business owners are a part of the team

There should not be such a fight between business and technology departments. But there almost always is.
Somebody has to be at fault and someone has to be right.

Fair enough.

But if you can learn something from negotiations, is that the sooner you get the other side to “work on your team” — the better. The quicker any negotiation will come to a reasonable resolution.

This should be much easier inside a company. After all you are all working on the same team, presuming that you all agree with the company’s vision and believe in the product. If not, that’s a different story.

To reiterate my earlier point, the sooner technology gets business owners into the process of development the better. If business owners do not have time, they need to designate a resources that would be responsible for being there all the way through. From idea to implementation.

Without such intertwining of business and technology it would be nearly impossible to establish clear communication and well delivered products.

6. If it feels too complicated, it probably is

If it can be tested and delivered it should be a user story.

Yes, a single function can and should be its own user story when appropriate.

Sometimes it is tempting to say that a chunk of code by itself is useless because it doesn’t do anything. Well, let me ask you again… can it be tested? Does it perform any sort of logic? Did you actually attempt do write any unit tests for it? Chances are it is not as useless as you think.

Let’s take a look at an example…

A user story might require us to create a table add a couple of methods for data processing to our model or DAO and display this data in the UI somewhere.

While this sounds like one cohesive unit it really is not.
If you take a TDD approach (as you should), you can break this up into smaller testable units.

– release script/migration to create the table
– unit test and fixture data for method one
– production code for method one
– unit test and fixture data for method two
– production code for method two
– ui design
– front-end test

That’s a lot of “testable” stuff that can come out of a single user story. And even if each one of those pieces cannot become its own user story, it can certainly be split up into at least two or three.

Even with everything above working systematically and perfectly we’ll never achieve a consistent perfect-world scenario, where there are no emergencies or otherwise re-scoped sprints. Things change and blow up and priorities shift.

As developer I need to make sure I am always aware of what I am supposed to deliver, and that I am relatively protected because I am not working on my own in the dark corner, but make my work transparent, demo-able and involve others to be aware of what is being done from the start of the process to the end.

To conclude, I’d like to address all the managers out there… If you have to ask: “What is going on with this story?”, or “Where is that story?” (other than in a daily stand-up) then you are a doing a poor job “managing”. Manager (scrum master, project manager) should be equipped with the necessary tools to answer that question. It is not developer’s job to report on the progress. Simply put if you are not aware of what’s going on with your team, then you need to seriously re-evaluate your approach to communication and your current business processes.

Work with multiple remotes and share your code with Git

It is pretty often that we need to work with a team of developers.
Sometimes we need to share our code; and with Git having the ability to work with multiple “remotes” is really convenient.

Think of the “remote” as simply the URL of the repo where you push your updates.

Let’s dive into an example…

Mike has been doing some development in his feature/adding-new-report branch.
It would be awesome for me to work in my own branch (as any good git user will do), yet I would also love to be able to easily sync our work.

Thankfully with some modern tools and “Pull requests” available on most git hosting sites (github, stash, bitbucket, etc.), we can do it very easily.
The method below would also work just fine without “pull requests” but for the simplicity and an extra step of sanity-checks, I recommend using PR’s when possible.

So now we can consider a typical setup.

You have a central repo at the remote (destination) called “origin”, where the code of your application lives. When we refer to “origin” in git, we are really talking about some destination (or URL) pointing to the git repo somewhere out there.
With that said, it is also not uncommon for each developer to clone the main repo, work on their own copy and then submit their changes by merging the changes in, or creating a “pull request”.

We can easily add other remotes besides “origin” (which is, by the way, just a name given to the initial repo destination. It can be changed, moved and you can add more destinations as we’ll see below).

Let me add Mike’s remote so I can checkout code from his repo (not just “origin”).

git remote add mike git@github.com:mike/my-first-repo.git

Now we have a new remote called “mike”

I can create a local branch to sync up with Mike’s changes:

git branch --track feature/adding-new-report mike/feature/adding-new-report

Now we have a local branch called feature/adding-new-report. It is “connected” (tracking) with Mike’s branch in his repo.
Pretty cool.

Now we should get the latest changes into our local branch. (Presuming we are now on our local feature/adding-new-report branch).

git pull mike feature/adding-new-report

(Got the latest code from Mike).

Now we can make any changes and commit them as usual.

At this point we have a couple of options. We could just push the changes up to Mike’s branch/repo.
Additionally (and preferably, in my opinion) we could do the following:

git push origin feature/adding-new-report

This will create the same branch in our own remote repo (presuming “origin” points to the URL of my development repo). This enables us to create a pull request, which will clearly show the differences between my changes and the current state of Mike’s branch. (if we were diligent about keeping things up-to-date only the necessary changes without conflicts should be shown in the diff.)

Overall there are a few benefits to using pull requests. You can easily see what was changed, the PR can be updated, based on review (you can add multiple reviewers) and it keep things a lot more cleaner and organized. (Dealing with conflicts is a bit easier).

For reference here’s github’s overview of working with Pull Requests.