Autotest

Autotest continuously scans the files in your project for changes and runs the appropriate tests. Test failures are run until they have all passed. Then the full test suite is run to ensure that nothing else was inadvertantly broken.

If you want Autotest to start over from the top, hit ^C once. If you want Autotest to quit, hit ^C twice.

Rails:

The autotest command will automatically discover a Rails directory by looking for config/environment.rb. When Rails is discovered, autotest uses RailsAutotest to perform file mappings and other work. See RailsAutotest for details.

Plugins:

Plugins are available by creating a .autotest file either in your project root or in your home directory. You can then write event handlers in the form of:

  Autotest.add_hook hook_name { |autotest| ... }

The available hooks are listed in ALL_HOOKS.

See example_dot_autotest.rb for more details.

If a hook returns a true value, it signals to autotest that the hook was handled and should not continue executing hooks.

Naming:

Autotest uses a simple naming scheme to figure out how to map implementation files to test files following the Test::Unit naming scheme.

Test files must be stored in test/
Test files names must start with test_
Test class names must start with Test
Implementation files must be stored in lib/
Implementation files must match up with a test file named test_.*implementation.rb

Strategy:

Find all files and associate them from impl <-> test.
Run all tests.
Scan for failures.
Detect changes in ANY (ruby?. file, rerun all failures + changed files.
Until 0 defects, goto 3.
When 0 defects, goto 2.

Attributes

known_files[W]

completed_re[RW]

extra_class_map[RW]

extra_files[RW]

failed_results_re[RW]

files_to_test[RW]

find_order[RW]

interrupted[RW]

last_mtime[RW]

libs[RW]

order[RW]

output[RW]

results[RW]

sleep[RW]

tainted[RW]

testlib[RW]

find_directories[RW]

unit_diff[RW]

wants_to_quit[RW]

Public Class Methods

add_discovery(&proc) click to toggle source

Add a proc to the collection of discovery procs. See autodiscover.

    # File lib/autotest.rb, line 81
81:   def self.add_discovery &proc
82:     @@discoveries << proc
83:   end

add_hook(name, &block) click to toggle source

Add the supplied block to the available hooks, with the given name.

     # File lib/autotest.rb, line 653
653:   def self.add_hook(name, &block)
654:     HOOKS[name] << block
655:   end

autodiscover() click to toggle source

Automatically find all potential autotest runner styles by searching your loadpath, vendor/plugins, and rubygems for “autotest/discover.rb“. If found, that file is loaded and it should register discovery procs with autotest using add_discovery. That proc should return one or more strings describing the user’s current environment. Those styles are then combined to dynamically invoke an autotest plugin to suite your environment. That plugin should define a subclass of Autotest with a corresponding name.

Process:

All autotest/discover.rb files loaded.
Those procs determine your styles (eg [“rails”, “rspec”]).
Require file by sorting styles and joining (eg ‘autotest/rails_rspec’).
Invoke run method on appropriate class (eg Autotest::RailsRspec.run).

Example autotest/discover.rb:

  Autotest.add_discovery do
    "rails" if File.exist? 'config/environment.rb'
  end

     # File lib/autotest.rb, line 110
110:   def self.autodiscover
111:     require 'rubygems'
112: 
113:     Gem.find_files("autotest/discover").each do |f|
114:       load f
115:     end
116: 
117:     @@discoveries.map { |proc| proc.call }.flatten.compact.sort.uniq
118:   end

new() click to toggle source

Initialize the instance and then load the user’s .autotest file, if any.

     # File lib/autotest.rb, line 150
150:   def initialize
151:     # these two are set directly because they're wrapped with
152:     # add/remove/clear accessor methods
153:     @exception_list = []
154:     @test_mappings = []
155: 
156:     self.completed_re = /\d+ tests, \d+ assertions, \d+ failures, \d+ errors/
157:     self.extra_class_map   = {}
158:     self.extra_files       = []
159:     self.failed_results_re = /^\s+\d+\) (?:Failure|Error):\n(.*?)\((.*?)\)/
160:     self.files_to_test     = new_hash_of_arrays
161:     self.find_order        = []
162:     self.known_files       = nil
163:     self.libs              = ]. lib test].join(File::PATH_SEPARATOR)
164:     self.order             = :random
165:     self.output            = $stderr
166:     self.sleep             = 1
167:     self.testlib           = "test/unit"
168:     self.find_directories  = ['.']
169:     self.unit_diff         = "unit_diff -u"
170: 
171:     self.add_mapping(/^lib\/.*\.rb$/) do |filename, _|
172:       possible = File.basename(filename).gsub '_', '_?'
173:       files_matching %^test/.*#{possible}$%
174:     end
175: 
176:     self.add_mapping(/^test.*\/test_.*rb$/) do |filename, _|
177:       filename
178:     end
179: 
180:     [File.expand_path('~/.autotest'), './.autotest'].each do |f|
181:       load f if File.exist? f
182:     end
183:   end

run() click to toggle source

Initialize and run the system.

     # File lib/autotest.rb, line 123
123:   def self.run
124:     new.run
125:   end

Public Instance Methods

add_exception(regexp) click to toggle source

Adds regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.

     # File lib/autotest.rb, line 583
583:   def add_exception regexp
584:     raise "exceptions already compiled" if defined? @exceptions
585: 
586:     @exception_list << regexp
587:     nil
588:   end

add_mapping(regexp, prepend = false, &proc) click to toggle source

Adds a file mapping, optionally prepending the mapping to the front of the list if prepend is true. regexp should match a file path in the codebase. proc is passed a matched filename and Regexp.last_match. proc should return an array of tests to run.

For example, if test_helper.rb is modified, rerun all tests:

  at.add_mapping(/test_helper.rb/) do |f, _|
    at.files_matching(/^test.*rb$/)
  end

     # File lib/autotest.rb, line 547
547:   def add_mapping(regexp, prepend = false, &proc)
548:     if prepend then
549:       @test_mappings.unshift [regexp, proc]
550:     else
551:       @test_mappings.push [regexp, proc]
552:     end
553:     nil
554:   end

add_sigint_handler() click to toggle source

Installs a sigint handler.

     # File lib/autotest.rb, line 279
279:   def add_sigint_handler
280:     trap 'INT' do
281:       if self.interrupted then
282:         self.wants_to_quit = true
283:       else
284:         unless hook :interrupt then
285:           puts "Interrupt a second time to quit"
286:           self.interrupted = true
287:           Kernel.sleep 1.5
288:         end
289:         raise Interrupt, nil # let the run loop catch it
290:       end
291:     end
292:   end

all_good() click to toggle source

If there are no files left to test (because they’ve all passed), then all is good.

     # File lib/autotest.rb, line 298
298:   def all_good
299:     files_to_test.empty?
300:   end

clear_exceptions() click to toggle source

Clears the list of exceptions for find_file. This must be called before the exceptions are compiled.

     # File lib/autotest.rb, line 604
604:   def clear_exceptions
605:     raise "exceptions already compiled" if defined? @exceptions
606:     @exception_list.clear
607:     nil
608:   end

clear_mappings() click to toggle source

Clears all file mappings. This is DANGEROUS as it entirely disables autotest. You must add at least one file mapping that does a good job of rerunning appropriate tests.

     # File lib/autotest.rb, line 571
571:   def clear_mappings
572:     @test_mappings.clear
573:     nil
574:   end

consolidate_failures(failed) click to toggle source

Returns a hash mapping a file name to the known failures for that file.

     # File lib/autotest.rb, line 319
319:   def consolidate_failures(failed)
320:     filters = new_hash_of_arrays
321: 
322:     class_map = Hash[*self.find_order.grep(/^test/).map { |f| # TODO: ugly
323:                        [path_to_classname(f), f]
324:                      }.flatten]
325:     class_map.merge!(self.extra_class_map)
326: 
327:     failed.each do |method, klass|
328:       if class_map.has_key? klass then
329:         filters[class_map[klass]] << method
330:       else
331:         output.puts "Unable to map class #{klass} to a file"
332:       end
333:     end
334: 
335:     return filters
336:   end

exceptions() click to toggle source

Return a compiled regexp of exceptions for find_files or nil if no filtering should take place. This regexp is generated from exception_list.

     # File lib/autotest.rb, line 615
615:   def exceptions
616:     unless defined? @exceptions then
617:       if @exception_list.empty? then
618:         @exceptions = nil
619:       else
620:         @exceptions = Regexp.union(*@exception_list)
621:       end
622:     end
623: 
624:     @exceptions
625:   end

files_matching(regexp) click to toggle source

Returns all known files in the codebase matching regexp.

     # File lib/autotest.rb, line 531
531:   def files_matching regexp
532:     self.find_order.select { |k| k =~ regexp }
533:   end

find_files() click to toggle source

Find the files to process, ignoring temporary files, source configuration management files, etc., and return a Hash mapping filename to modification time.

     # File lib/autotest.rb, line 343
343:   def find_files
344:     result = {}
345:     targets = self.find_directories + self.extra_files
346:     self.find_order.clear
347: 
348:     targets.each do |target|
349:       order = []
350:       Find.find(target) do |f|
351:         Find.prune if f =~ self.exceptions
352: 
353:         next if test dd, f
354:         next if f =~ /(swp|~|rej|orig)$/ # temporary/patch files
355:         next if f =~ /\/\.?#/            # Emacs autosave/cvs merge files
356: 
357:         filename = f.sub(/^\.\//, '')
358: 
359:         result[filename] = File.stat(filename).mtime rescue next
360:         order << filename
361:       end
362:       self.find_order.push(*order.sort)
363:     end
364: 
365:     return result
366:   end

find_files_to_test(files=find_files) click to toggle source

Find the files which have been modified, update the recorded timestamps, and use this to update the files to test. Returns true if any file is newer than the previously recorded most recent file.

     # File lib/autotest.rb, line 374
374:   def find_files_to_test(files=find_files)
375:     updated = files.select { |filename, mtime| self.last_mtime < mtime }
376: 
377:     p updated if $v unless updated.empty? || self.last_mtime.to_i == 0
378: 
379:     hook :updated, updated unless updated.empty? || self.last_mtime.to_i == 0
380: 
381:     updated.map { |f,m| test_files_for(f) }.flatten.uniq.each do |filename|
382:       self.files_to_test[filename] # creates key with default value
383:     end
384: 
385:     if updated.empty? then
386:       nil
387:     else
388:       files.values.max
389:     end
390:   end

get_to_green() click to toggle source

Keep running the tests after a change, until all pass.

     # File lib/autotest.rb, line 218
218:   def get_to_green
219:     begin
220:       run_tests
221:       wait_for_changes unless all_good
222:     end until all_good
223:   end

handle_results(results) click to toggle source

Check results for failures, set the “bar” to red or green, and if there are failures record this.

     # File lib/autotest.rb, line 396
396:   def handle_results(results)
397:     failed = results.scan(self.failed_results_re)
398:     completed = results =~ self.completed_re
399: 
400:     self.files_to_test = consolidate_failures failed if completed
401: 
402:     color = completed && self.files_to_test.empty? ? :green : :red
403:     hook color unless $TESTING
404: 
405:     self.tainted = true unless self.files_to_test.empty?
406:   end

hook(name, *args) click to toggle source

Call the event hook named name, executing all registered hooks until one returns true. Returns false if no hook handled the event.

     # File lib/autotest.rb, line 635
635:   def hook(name, *args)
636:     deprecated = {
637:       # none currently
638:     }
639: 
640:     if deprecated[name] and not HOOKS[name].empty? then
641:       warn "hook #{name} has been deprecated, use #{deprecated[name]}"
642:     end
643: 
644:     HOOKS[name].any? do |plugin|
645:       plugin[self, *args]
646:     end
647:   end

known_files() click to toggle source

Lazy accessor for the known_files hash.

     # File lib/autotest.rb, line 411
411:   def known_files
412:     unless @known_files then
413:       @known_files = Hash[*find_order.map { |f| [f, true] }.flatten]
414:     end
415:     @known_files
416:   end

make_test_cmd(files_to_test) click to toggle source

Generate the commands to test the supplied files

     # File lib/autotest.rb, line 421
421:   def make_test_cmd files_to_test
422:     cmds = []
423:     full, partial = reorder(files_to_test).partition { |k,v| v.empty? }
424:     base_cmd = "#{ruby} -I#{libs} -rubygems"
425: 
426:     unless full.empty? then
427:       classes = full.map {|k,v| k}.flatten.uniq
428:       classes.unshift testlib
429:       cmds << "#{base_cmd} -e \"%w[#{classes.join(' ')}].each { |f| require f }\" | #{unit_diff}"
430:     end
431: 
432:     partial.each do |klass, methods|
433:       regexp = Regexp.union(*methods).source
434:       cmds << "#{base_cmd} #{klass} -n \"/^(#{regexp})$/\" | #{unit_diff}"
435:     end
436: 
437:     return cmds.join("#{SEP} ")
438:   end

new_hash_of_arrays() click to toggle source

     # File lib/autotest.rb, line 440
440:   def new_hash_of_arrays
441:     Hash.new { |h,k| h[k] = [] }
442:   end

path_to_classname(s) click to toggle source

Convert a path in a string, s, into a class name, changing underscores to CamelCase, etc.

     # File lib/autotest.rb, line 306
306:   def path_to_classname(s)
307:     sep = File::SEPARATOR
308:     f = s.sub(/^test#{sep}/, '').sub(/\.rb$/, '').split(sep)
309:     f = f.map { |path| path.split(/_|(\d+)/).map { |seg| seg.capitalize }.join }
310:     f = f.map { |path| path =~ /^Test/ ? path : "Test#{path}"  }
311: 
312:     f.join('::')
313:   end

remove_exception(regexp) click to toggle source

Removes regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.

     # File lib/autotest.rb, line 594
594:   def remove_exception regexp
595:     raise "exceptions already compiled" if defined? @exceptions
596:     @exception_list.delete regexp
597:     nil
598:   end

remove_mapping(regexp) click to toggle source

Removed a file mapping matching regexp.

     # File lib/autotest.rb, line 559
559:   def remove_mapping regexp
560:     @test_mappings.delete_if do |k,v|
561:       k == regexp
562:     end
563:     nil
564:   end

reorder(files_to_test) click to toggle source

     # File lib/autotest.rb, line 444
444:   def reorder files_to_test
445:     case self.order
446:     when :alpha then
447:       files_to_test.sort_by { |k,v| k }
448:     when :reverse then
449:       files_to_test.sort_by { |k,v| k }.reverse
450:     when :random then
451:       max = files_to_test.size
452:       files_to_test.sort_by { |k,v| rand(max) }
453:     when :natural then
454:       (self.find_order & files_to_test.keys).map { |f| [f, files_to_test[f]] }
455:     else
456:       raise "unknown order type: #{self.order.inspect}"
457:     end
458:   end

rerun_all_tests() click to toggle source

Rerun the tests from cold (reset state)

     # File lib/autotest.rb, line 463
463:   def rerun_all_tests
464:     reset
465:     run_tests
466: 
467:     hook :all_good if all_good
468:   end

reset() click to toggle source

Clear all state information about test failures and whether interrupts will kill autotest.

     # File lib/autotest.rb, line 474
474:   def reset
475:     self.files_to_test.clear
476:     self.find_order.clear
477:     self.interrupted = false
478:     self.known_files = nil
479:     self.last_mtime = T0
480:     self.tainted = false
481:     self.wants_to_quit = false
482: 
483:     hook :reset
484:   end

ruby() click to toggle source

Determine and return the path of the ruby executable.

     # File lib/autotest.rb, line 489
489:   def ruby
490:     ruby = ENV['RUBY']
491:     ruby ||= File.join(Config::CONFIG['bindir'],
492:                        Config::CONFIG['ruby_install_name'])
493: 
494:     ruby.gsub! File::SEPARATOR, File::ALT_SEPARATOR if File::ALT_SEPARATOR
495: 
496:     return ruby
497:   end

run() click to toggle source

Repeatedly run failed tests, then all tests, then wait for changes and carry on until killed.

     # File lib/autotest.rb, line 189
189:   def run
190:     hook :initialize
191:     reset
192:     add_sigint_handler
193: 
194:     self.last_mtime = Time.now if $f
195: 
196:     loop do # ^c handler
197:       begin
198:         get_to_green
199:         if self.tainted then
200:           rerun_all_tests
201:         else
202:           hook :all_good
203:         end
204:         wait_for_changes
205:       rescue Interrupt
206:         break if self.wants_to_quit
207:         reset
208:       end
209:     end
210:     hook :quit
211:   rescue Exception => err
212:     hook :died, err
213:   end

run_tests() click to toggle source

Look for files to test then run the tests and handle the results.

     # File lib/autotest.rb, line 228
228:   def run_tests
229:     hook :run_command
230: 
231:     new_mtime = self.find_files_to_test
232:     return unless new_mtime
233:     self.last_mtime = new_mtime
234: 
235:     cmd = self.make_test_cmd self.files_to_test
236:     return if cmd.empty?
237: 
238:     puts cmd unless $q
239: 
240:     old_sync = $stdout.sync
241:     $stdout.sync = true
242:     self.results = []
243:     line = []
244:     begin
245:       open("| #{cmd}", "r") do |f|
246:         until f.eof? do
247:           c = f.getc or break
248:           if RUBY19 then
249:             print c
250:           else
251:             putc c
252:           end
253:           line << c
254:           if c == \n\ then
255:             self.results << if RUBY19 then
256:                               line.join
257:                             else
258:                               line.pack "c*"
259:                             end
260:             line.clear
261:           end
262:         end
263:       end
264:     ensure
265:       $stdout.sync = old_sync
266:     end
267:     hook :ran_command
268:     self.results = self.results.join
269: 
270:     handle_results(self.results)
271:   end

test_files_for(filename) click to toggle source

Return the name of the file with the tests for filename by finding a test_mapping that matches the file and executing the mapping’s proc.

     # File lib/autotest.rb, line 504
504:   def test_files_for(filename)
505:     result = @test_mappings.find { |file_re, ignored| filename =~ file_re }
506: 
507:     p :test_file_for => [filename, result.first] if result and $DEBUG
508: 
509:     result = result.nil? ? [] : [result.last.call(filename, $~)].flatten
510: 
511:     output.puts "No tests matched #{filename}" if
512:       ($v or $TESTING) and result.empty?
513: 
514:     result.sort.uniq.select { |f| known_files[f] }
515:   end

wait_for_changes() click to toggle source

Sleep then look for files to test, until there are some.

     # File lib/autotest.rb, line 520
520:   def wait_for_changes
521:     hook :waiting
522:     Kernel.sleep self.sleep until find_files_to_test
523:   end

Home Classes Methods

In Files

Parent

Namespace

Methods

Files