taskRunner
taskRunner is a sample set of stacks to demonstrate the use of socket communication to enable Livecode applications to utilize multiple cores or multiple processors. Nowadays, most computers (even laptops and tablets) come with multiple cores or processors. Livecode is currently only single-threaded, so often I see that 3 out of the 4 cores on my MacBook Pro are idle, while I am waiting for some lengthy operation to complete.
Many operations that an application needs to carry out can be broken down into a number of smaller, more-or-less independent tasks. taskRunner provides a structure to make it (relatively) easy to parcel out these individual tasks to separate processes, which will then be run on the multiple cores.
Examples of operations which can be easily broken down to independent, or partly independent, tasks include:
- indexing files (e.g. calculating md5hash of each file to determine if copies of folder are in sync)
- resizing images (or other image operations)
- URL link checking
The first two of those examples are very simple - each file can be handled separately, so each task is completely independent. The only interesting design decision is whether to make each file a separate task, or make the processing of each folder a task. More interesting is URL checking, because given an initial target URL, the processing of that URL will generate a list of other URL which should also be checked. Thus although each task is largely independent of any other task, it may generate additional tasks to be performed.
There are two main parts to this demonstration sample:
- taskRunner - a stack which runs (normally as a standalone app) and actually runs the tasks
- taskClientLib - a library stack to be incorporated into your app to requests tasks to run
In addition, there are some sample stacks:
- test-task-runner - very simple stack which allows you to run ome of the samples
- verybusy - stack to use up CPU, purely as a demonstration of multi-processing
- IndexFiles - stack to index a directory of files
- CheckURL - stack to check a URL, returning a list of broken links, and also any other URL which should be checked
All the files should be downloaded via the individual links above.
Real Soon Now (tm) I will get some documentation together on how these all work, but in fact they are relatively straightforward, so feel free to download them and play around. The two main stacks can both be loaded into the IDE and developed/tested there, but of course this puts them into the same single-threaded process, so to actually get any benefit, you need to build taskRunner as a standalone. This isn't usually a problem - taskRunner itself is pretty simple and shouldn't need too much changing; the majority of the development you'll normally need to do is on the stack which performs the task, and that will be loaded each time the client requests it, so there is no great issue with having taskRunner be a standalone. The main client app, which calls on taskClientLib to request that stacks be loaded and that tasks be performed on its behalf will often be in the IDE under development. Just make sure that you have an easy way to re-request stack loading, to ensure that any changes you make in the client stack can be re-loaded.