Include a little stopwatch next to the task (where the notes/attachments icons are) in order to start a timer to keep track of how much time was spent on that specific task.

And a place where we can see how much time was spent on each one.