#17 open

Migrations should be consistent with Rails' mechanism

Reported by Tekin | June 5th, 2008 @ 07:46 AM | in 2.1.2

I'm running Rails 2.1 and the latest master of engines from github and I'm having troubles with plugin migrations.

The initial set of migrations run fine, but when I update a plugin and try and migrate up to the latest version, Rails ignores the plugin_schema version and tries to run all plugin migrations from the top.

I've noticed the rake tasks for migrating plugins and tried those but with the same results.

Comments and changes to this ticket

  • James Adam

    James Adam June 6th, 2008 @ 05:31 AM

    (from [ec608c0d53144646609925023cb6108fe5a53458]) Added a test for incremental migration, to try and capture the issue described by [#17], but without joy.


  • James Adam

    James Adam June 6th, 2008 @ 05:46 AM

    • State changed from “new” to “more_details_needed”
  • Tekin

    Tekin June 9th, 2008 @ 01:25 AM

    Test runs fine, will investigate further as migrations were running fine for my plugin on rails 1.2.x

  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 05:01 AM

    I had the same problem:

    $ script/generate plugin_migration


    • config.breakpoint_server has been deprecated and has no effect. *


    exists db/migrate

    create db/migrate/20080610142419_page_engine_to_version_20080610141245.rb

    $ rake db:migrate

    (in .....)

    == 20080610142419 PageEngineToVersion20080610141245: migrating ================

    == 1 CreateLinks: migrating ===================================================

    -- create_table(:links)

    rake aborted!

    SQLite3::SQLException: table "links" already exists: CREATE TABLE "links" (.......)

    (See full trace by running task with --trace)

    $ cd vendor/plugins/engines/

    $ svn update

    At revision 705.

    You can quite clearly see it is trying to migrate the first migration for that plugin.



  • Tekin

    Tekin June 10th, 2008 @ 05:10 AM

    I've created a fork with a failing test:


    It only seems to fail once there are more than two migrations.

  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 05:40 AM

    Hi, I've found a way to fix this problem:

    So...... the schema for storing versions is slightly different:

    CREATE TABLE plugin_schema_info (plugin_name varchar(255), version integer);

    For normal rails, is actually a string not an integer:

    CREATE TABLE "schema_migrations" ("version" varchar(255) NOT NULL);

    Well, I looked at that table:

    sqlite> select * from schema_migrations;












    so... every version is listed... so, I did the same in the plugin schema table:

    sqlite> select * from plugin_schema_info;





    sqlite> insert into plugin_schema_info values("page_engine", 1);

    sqlite> insert into plugin_schema_info values("page_engine", 2);

    sqlite> insert into plugin_schema_info values("page_engine", 3);

    Once I done this it correctly try to migrate starting from schema version 4...

    $ rake db:migrate

    (in /Users/samuel/Sites/Development/www.stmargarets.school.nz)

    == 20080610142419 PageEngineToVersion20080610141245: migrating ================

    == 4 CreateSections: migrating ================================================

    -- create_table(:sections)

    rake aborted!

    SQLite3::SQLException: table "sections" already exists: CREATE TABLE "sections" ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, "page_id" integer DEFAULT NULL NULL, "position" integer DEFAULT NULL NULL, "content" text DEFAULT NULL NULL, "markup_as" varchar(255) DEFAULT NULL NULL)

    So......... I think the correct behaviour in this case is possibly an upgrade rake file which inserts these dummy entries for projects that already exist, and the migration code needs to add these individually for migrations done from now.

    I'm not familiar with the code to write this patch quickly. Let me know if there is some other way I can help.



  • James Adam

    James Adam June 10th, 2008 @ 05:42 AM

    • Milestone set to 2.1.2
    • State changed from “more_details_needed” to “open”

    Tekin - thanks! Failing tests really help me track down what's going on.

    I've committed a potential fix: http://github.com/lazyatom/engin...

    Can you check if that fixes your issues? Thanks!

  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 05:44 AM

    After I did a successful migration, my table was completely munted:

    sqlite> select * from plugin_schema_info;















  • James Adam

    James Adam June 10th, 2008 @ 05:45 AM

    Samuel - interesting. We could start using the plugin schema table like that (with a row for each migration, rather than a row for each plugin). That's definitely possible.

    The change I committed is slightly simpler, but somewhat 'naughty' - it fakes the set of migrated versions by just checking the current version, and reproducing an array from 1 up to that number.

    Sven Fuchs has a branch that converts the plugin migrations to be more like the new system in Rails - I guess it's a case of deciding whether or not the benefits are there at the moment.

  • James Adam

    James Adam June 10th, 2008 @ 05:48 AM

    How did you generate the migration within your plugin? Looks like it has a Rails 2.0-ish timestamp as the version, rather than a simple integer.

    Thinking about this more, if people are moving migrations from Rails 2 applications into plugins, we'll need to support this better. Can you try merging Sven Fuchs fork and see if that helps?

  • Tekin

    Tekin June 10th, 2008 @ 06:18 AM

    I think the closer engines follows Rails the better so a big +1 for having time stamped migrations and the plugin schema table keep track of the applied migrations as Rails does.

    As Samual said, we'd need a simple rake task to possibly rename an existing set of migrations with timestamps and update the plugin schema table with the migrations that have been run in the new format..

    In this case. it would also be handy to have a generator to create migrations in your plugins with time stamps.

    I'm willing to help with any of this, let me know and I'll have a route about in the source.

  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 06:20 AM

    Here is a patch that fixes everything... world hunger, crime, the works.

    Actually, this patch just modifies the Engines to use plugins like the standard rails 2.1 behaviour.



  • Samuel Williams
  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 06:21 AM

    Sorry, I couldn't see where the attachments were going, they are up on the top right hand side for anyone who can't figure it out.

    I'm stupid.. its almost 5am.. so please forgive me!

  • James Adam

    James Adam June 10th, 2008 @ 06:27 AM

    • Assigned user set to “Sven Fuchs”

    The patch looks OK, although I'd be curious to see how it compares to Sven's fork.

    What's missing are:

    • appropriate changes to the tests
    • some documentation or code that deals with upgrading

    Any takers?

  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 06:32 AM

    • Assigned user cleared.

    I have one more issue....... version is not a string as per the main rails schema info table... is this a problem going forward?

    Where can I see Sven's fork?

  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 06:32 AM

    • Assigned user set to “Sven Fuchs”

    Sorry, I didn't mean to change the assigned user, it just chose no one by default... weird

  • James Adam

    James Adam June 10th, 2008 @ 06:39 AM

    • Assigned user changed from “Sven Fuchs” to “James Adam”

    Development is on github, and you can find Sven's fork via the network: http://github.com/svenfuchs/engines

    Oh, and I only really set the user to Sven so he'd start getting notifications of this ticket :)

  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 06:49 AM

    Um, here is a patch to fix old migrations... but I think it might be worth changing the column from int to varchar(255) too...

  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 06:51 AM

    By the way, I haven't actually tested it much.. I only had one place where it was needed, but I wrote it for completeness

  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 07:05 AM

    Sven's fork looks fairly similar to what I proposed, however I think his patch is slightly more complete, I just threw this one together quickly. It might be worth it if Sven takes a look at the patches and sees if there is anything to add to his fork, but his fork looks good. Its funny that we came up with very similar codes.



  • Samuel Williams

    Samuel Williams June 10th, 2008 @ 07:06 AM

    The last remaining issue is whether or not to keep the version column as int, or change it to string like standard Rails.. I think that is an important issue to consider.

  • Tekin

    Tekin June 12th, 2008 @ 07:04 AM

    Sven's fork is definitely looking promising to me.

    I think it would be necessary to have a rake task to initialise the new plugin_schema_migrations table with any migrations that have already been run using the current system.

    With that, and some clear documentation, I think everyone's a winner!

  • Cameron Yule

    Cameron Yule July 15th, 2008 @ 01:38 AM

    • Tag set to major, migrations

    Was caught out by this bug as well - I'd generated some migrations and moved them into my plugin, generated a plugin_migration and ran migrate - it would always stall after the first migration.

    Looking forward to 2.1.1 and a fix! :)

  • Cameron Yule

    Cameron Yule July 15th, 2008 @ 01:39 AM

    • Tag cleared.

    Erk, didn't mean to change those tags :S

  • Tekin

    Tekin July 15th, 2008 @ 02:07 AM

    Hi Cameron,

    Try Sven's fork on github - it works out of the box with Rails 2.1, but I don't think it works with edge.

    If you are using pre-existing plugins, you will have to rename all your migrations to make sure there are no name clashes between any of the migrations (all migrations need a unique name, but not necessarily timestamped..)

  • Samuel Williams

    Samuel Williams July 15th, 2008 @ 05:03 AM

    Hi Cameron, you can also use the patch against the current engines. I've used it for a couple of my projects and it works well. I've put the latest version up for you, if you want to use it.

    Tekin; why should you need to rename all your migrations? with my patch, that isn't needed at all, afaik, and everything works just fine.

  • Tekin

    Tekin July 15th, 2008 @ 07:07 AM

    Hi Samuel,

    My comment was against Sven's fork.

    I had problems running migrations from more than one plugin. I'm not entirely sure, but if I remember correctly, when migrations across two plugins had a number/timestamp clash, the second migration wasn't getting run. I think I tracked it down to the way Sven's fork checks if a migration has been run and removing the duplicate numbers/timestamps resolved the issue.

  • Samuel Williams

    Samuel Williams July 15th, 2008 @ 07:55 AM

    Thanks for that clarification.

    I suspect my patch is more typical than what Sven is doing, but without spending time looking at his code, I couldn't say for sure. What I can say is that my patch makes engine rails migrations function almost identically to the way the function for the "core" database model in Rails.

    For example, in the main rails schema table, each migration is listed once - its version number. In my patch, I have copied this behavior, but instead of just the version number, there is also the name of the plugin. This is why you need to run the rake task to add all the previous entries into the engines schema table. What it does is get the latest number for each entry in the engines schema, and then counts from 0..n and adds another entry into the table. Then, the rest of the functionality is exactly the same as core rails.

    So, I like my patch and it works well. I've used it on a number of commercial sites with no gotchas or anything. Just use the patch, run the rake task, and everything should be sweet ^_^ (touch wood :)

    Kind regards,


  • Sven Fuchs

    Sven Fuchs July 15th, 2008 @ 08:00 AM

    "I suspect my patch is more typical than what Sven is doing, but without spending time looking at his code, I couldn't say for sure."

    No idea what you mean by that, Samuel, but IIRC I've just copied what ActiveRecord is doing to migrate to the timestamped format. :)

  • Samuel Williams

    Samuel Williams July 15th, 2008 @ 08:08 AM

    Sorry, one more thing - to use the patch, copy it to the vendor/plugins directory

    The run

    myproject/vendor/plugins/ $ patch -p0 < engines.diff

    After that, run

    rake db:migrate:fix_engines_migrations

    Everything should be good to go.

    (Obviously, test things out before running it on a live site, no warranty, yada yada..)

  • Samuel Williams

    Samuel Williams July 15th, 2008 @ 08:10 AM

    Sven, based on Tekin's comments and my experience, I'm not sure why he is having a problem. Thats why I wonder what your implementation is doing. Obviously, without looking at the code and Tekin's problem more in depth, it is difficult to tell, however I've had no such problems with my patch.

  • Samuel Williams

    Samuel Williams July 15th, 2008 @ 08:12 AM

    Directly from the above patch:

    "INSERT INTO #{sm_table} (plugin_name, version) VALUES ('#{plugin_name}', '#{version}')"

    I am using the plugin_name and the version as the key values in the engines schema table. It shouldn't be a problem if migrations have the same name. Everything else just works as per core Rails.

  • Samuel Williams

    Samuel Williams July 15th, 2008 @ 07:31 PM

    Sven, I briefly looked at your codes just now. We appear to be doing the same thing, which means that the solution is probably the right one. I think your code looks better than mine, but I'm happy with what I put together for now.

    Sven, I don't know if you approached the problem of fixing the previous migration table. This is something that my patch adds a rake task for. How did you approach migrating a previous schema table? Is this something we could integrate into your git if you haven't approached this problem yet? My rake task I've used it several times with success and no times with trouble.

    Tekin, would you care to elaborate on the issues you are having with Sven's patch? It might be something that needs to be resolved. I haven't had any naming issues, so it would be good if you could walk us through a case where it doesn't work, so it can be investigated.

    Kind regards,


  • Tekin

    Tekin July 16th, 2008 @ 04:36 AM

    Hi Samuel/Sven, I can't specifically remember exactly what the problem was, all I can remember is that a fresh rails 2.1 application with my four engines enabled plugins was failing to migrate correctly. Specifically, the first plugin migrated fine, but the second one started migrating from the version number after the number of migrations in the first. At the time, I assumed that this was because the mirgations were not timestamped, so I changed the file names and that fixed it, but only when all timestamps were unique across the plugins.

    If I get the chance later today, I'll try and recreate the problem and report back. It may have been me - looking at the code for Engines::Plugin::Migrator, I can't see why it would have done this.

    Sven, on another note, your fork has not been rebased with lazyatom/master in a while, any chance of doing this to bring across the latest changes that James has made?

  • Tekin

    Tekin July 16th, 2008 @ 07:19 AM

    Hi Sven/Samuel,

    I've had a little root around and it seems the problem is there, but you'll only see it if you have more than one plugin with migrations.

    Engines::Plugin::Migrator.migrate_plugin calls ActiveRecord::Migrator.migrate, which doesn't check against the plugin_name, just the version: looking at the sql log, I get

    "SELECT version FROM plugin_schema_migrations"

    This would explain why the second plugins migrations would kick in from the version number corresponding with the first plugins latest migration.

    I coudn't get your patch to work against the latest version of lazyatom/engines Samuel, I guess it's out of sync with the latest master, but I should imagine this will affect your patch too.

  • azimux

    azimux July 19th, 2008 @ 01:35 PM

    the way I have been getting around it for now is this: http://github.com/azimux/engines...

    Which is really simple but doesn't have any reverse compatibility with old engines plugin migrations.

    This works just fine as long as there's no timestamp clashes. It basically just prevents engines from loading it's migrator, while keeping the useful db:migrate:plugin(s) available and functioning properly.

    This works great for me and seems cleaner than the generator way of doing it, provided you're using it on a new project with timestamp based migrations from the start (and using plugins that have timestamp migrations as well.)

  • Tekin

    Tekin July 21st, 2008 @ 04:03 AM

    Azimux, that is an interesting approach.. In theory, with Rails 2.1 migrations, we do not need a plugin_schema_migrations table at all as the whole point of timestamped migrations is that they will (mostly!) be unique.

    I'm sure I remember reading that the new Rails migrator works such that it will run any migrations that have not been run, even if their timestamp is lower than the current highest timestamp, but I may be wrong here.

    If that is the case, we can simply update the Rails migrator to check the plugin migrations paths as well as the standard migrations directory and all will be hunky dory!

  • Tekin

    Tekin July 21st, 2008 @ 04:09 AM

    OK, I've checked and I was right, it will run any migrations that do not have an entry in the schema_migrations table, regardless of the timestamp. I guess it wouldn't work very well if it didn't!

    Anyway, this means the whole migration process for engines can be massivly simplified for Rails 2.1. I'll try and create a patch...

  • Tekin

    Tekin July 21st, 2008 @ 07:05 AM

    OK, I've done a quick proof of concept, it works in a similar fashion to your fork Azimux, but it keeps the plugin migration generator.

    Patch attached and you can see my fork at http://github.com/tekin/engines/....

    I did think about dropping the generator as you have, but I can see a case for keeping them - it keeps your app and plugin migrations separate so you can roll back changes in a more controlled fashion.

    I haven't fully tested this yet, but I thought I'd put the code out there for discussion. I'll get the tests updated and try it out on a couple of apps to see how it fairs.

  • Tekin

    Tekin July 21st, 2008 @ 12:32 PM

    Long rambling post over at the engines developers list regarding this issue. Any comments?

  • subimage

    subimage August 17th, 2008 @ 08:03 PM

    Very interested in the status of this as well.

    I'm experiencing the same problem with older migrations on Substruct.

    What's the best way to proceed?

  • subimage

    subimage August 22nd, 2008 @ 11:50 AM

    @Samuel thank you for the...

    rake db:migrate:fix_engines_migrations


    I tried it out on Substruct and it's working perfectly!

    Can we see this in the main branch anytime soon?

  • subimage

    subimage August 22nd, 2008 @ 11:52 AM

    PS: Consider that my +1 for Samuel's patch.

  • Samuel Williams

    Samuel Williams September 7th, 2008 @ 03:22 PM

    Hi Guys,

    I'm currently finalizing a version of my patch.

    I'm investigating the issue raised by Tekin.

    Regards, Samuel

  • Samuel Williams

    Samuel Williams September 7th, 2008 @ 03:54 PM

    While looking over Tekin's patch, I've noticed another way to resolve things.

    Currently, we have two schema info tables... one is keyed on soley on "datetime", and our engines info "datetime" and "plugin_name"

    I've noticed that the new Rails 2.1 info table is based on a varchar(255) which means that potentially, we can serialize both the datetime and plugin_name into the same table. This would mean that we only need one table, for example:

    For regular rails migrations: "20080610141245"

    For plugin migrations: "20080610141245-MyPlugin"

    These would be in the same table, rather than the current mode of having a separate table.

    However, I feel a bit mixed about this approach. Having a separate table allows us to be more resilient to changes in rails core. On the other hand, it means that things are a bit simpler from an implementation point of view, because more code of rails can be utilized. I think ultimately that because it is less code it might be better.

    I'm going to work on a patch.

    Any comments?

  • Tekin

    Tekin September 7th, 2008 @ 10:25 PM

    Something worth being aware of is that as of edge/2.2 Rails, you can turn timestamped migrations off with config.active_record.timestamped_migrations = false. This shouldn't cause any issues with what you are proposing as in either case, the schema migrations table is still created with a varchar for the version column.

    +1 - I like the solution you propose and it would work with both timestamped and numbered migrations.

    Should we not just accept that Rails is a moving target and that future versions of Rails may/will break engines? I think we are tying our hands too much trying to keep backwards compatibility. If releases are tagged correctly against the different versions of Rails, with clear instructions in the README to install the correct one, all should be dandy! boot.rb could be tweaked to raise an exception if the wrong version is installed.

    It shouldn't be too much work to create tags for versions 1.2, 2.0, 2.1 and 2.2 of Rails and would mean everyone is happy now and into the future.

    Any comments?

    Tekin Suleyman

    On 8 Sep 2008, at 02:54, Lighthouse wrote:

  • Samuel Williams

    Samuel Williams September 7th, 2008 @ 11:40 PM

    I have got a patch, but I'm working on a complete test case for Rails 2.1

    I agree, a moving target can be hard to hit, but I think if we do think about it (as we have been) we can produce a solution that is easier to manage.

    Regards, Samuel

  • Samuel Williams

    Samuel Williams September 8th, 2008 @ 12:13 AM



    is against Sven's fork, not engines r705. Would you be able to make a patch against 705?

    Kind regards, Samuel

  • Samuel Williams

    Samuel Williams September 8th, 2008 @ 12:20 AM


    I might be incorrect in the above post, however I can't appear to apply that patch to 705:

    Mochi:engines samuel$ patch -p1 < drop-plugin-schema-migrations-in-favour-of-using-schema-migrations.patch patching file generators/plugin_migration/plugin_migration_generator.rb patching file lib/engines/plugin.rb patching file lib/engines/plugin/migrator.rb Hunk #1 FAILED at 15. 1 out of 1 hunk FAILED -- saving rejects to file lib/engines/plugin/migrator.rb.rej patching file lib/engines/rails_extensions/migrations.rb

    Any ideas?

    Regards, Samuel

  • Tekin

    Tekin September 8th, 2008 @ 12:45 AM

    I'm not sure, I've just run the patch against a fresh clone of engines master and it applied fine..

  • Samuel Williams

    Samuel Williams September 8th, 2008 @ 02:57 PM

    Hi Guys,

    I've put together a patch which I would consider suitable for production use. This involves some stuff from my original patch, some stuff from Tekin's patch, and some new ideas for integrating directly into the rails schema info table.

    I've put together a test case rails app, which I'll upload next (I can't seem to add more than one file at a time).

    The test rails app has two test "engines" Engine1 and Engine2. Each engine has a mixture of migrations. I've tested most scenarios and it all seems to work as expected.

    In the example app, there are two old style dbs. There is one called "pre.sqlite3" which contains the old format of having a single number for each plugin, and "post.sqlite3" which has the "fix_engines_migrations" of the previous patch - one number (or date) for each migration.

    This patch includes a rake task to fix up all previous schema info formats:

    rake db:migrate:upgrade_plugin_migrations

    This will update Rails < 2.0, Rails + fix_engines_migration, and put all migrations into the main rails schema info table in the format "#{version}-#{plugin_name}"

    Can some people please test this?

  • Samuel Williams

    Samuel Williams September 8th, 2008 @ 02:58 PM

    Here is the example rails app with test databases for migrations, and two example engines with different kinds of migrations.

  • Samuel Williams

    Samuel Williams September 8th, 2008 @ 03:00 PM

    Sven, would you have some time to review this patch?

    Thanks for everyone's input.

  • Samuel Williams

    Samuel Williams September 8th, 2008 @ 04:36 PM

    Sorry, I've had to re-upload the patches.

  • Samuel Williams
  • Samuel Williams
  • Tekin

    Tekin September 9th, 2008 @ 12:00 AM

    Hi Samuel, will try take a look later

  • Samuel Williams

    Samuel Williams September 9th, 2008 @ 11:27 AM

    I would like to add some test cases for this patch. I've made a test app, but what would be the best way to create test cases? Because the scope of the patch is quite "big", it isn't just a single thing to test - we need to call rake tasks, etc to test. Is this possible?

    Also, if other people can try out this patch it would be a huge help.

    Thanks for everyone's time.

  • Samuel Williams

    Samuel Williams September 16th, 2008 @ 08:44 PM


    Do you have time to look at this patch? What is the process for getting this integrated?

    Kind regards, Samuel

  • James Adam

    James Adam September 17th, 2008 @ 03:24 AM

    Samuel's patch seems to be the simplest approach, which is definitely favourable.

    I've made this patch available in a branch on the main engines github repo: http://github.com/lazyatom/engin...

    Please test it out!

  • Tekin

    Tekin September 18th, 2008 @ 10:17 AM


    I've tested out the branch on both a fresh app and upgrading an existing app and in both cases everything ran as expected so that's a +1 from me.

    I've got a couple of patches, the first of which is attached. It takes a feature from Sven's fork that makes sure the generator doesn't create migrations with filenames that are too long.

  • Tekin

    Tekin September 18th, 2008 @ 10:21 AM

    Here's the second patch, this one modifies the rake task so that it also upgrades apps that have been built on Sven's fork.

  • Samuel Williams

    Samuel Williams October 6th, 2008 @ 10:45 AM


    Whats the deal with this patch, when will it be integrated?

    Kind regards, Samuel

  • Samuel Williams

    Samuel Williams October 6th, 2008 @ 05:10 PM


    I couldn't seem to apply your patches:

    Mochi:plugins samuel$ patch -p0 < 0001-Borrowed-feature-from-Sven-s-fork-that-ensures-gener.patch.txt can't find file to patch at input line 14 Perhaps you used the wrong -p or --strip option?

    The text leading up to this was:

    |From 727209762a716d01582433951693bd6d0819d7aa Mon Sep 17 00:00:00 2001 |From: Tekin Suleyman tekin@tekin.co.uk |Date: Thu, 18 Sep 2008 16:21:38 +0100 |Subject: [PATCH] Borrowed feature from Sven's fork that ensures generated plugin migrations do not exceed the maximum filename length supported by NTFS. | |--- | .../plugin_migration/plugin_migration_generator.rb | 24 ++++++++++++++++++- | 1 files changed, 22 insertions(+), 2 deletions(-) | |diff --git a/generators/plugin_migration/plugin_migration_generator.rb b/generators/plugin_migration/plugin_migration_generator.rb |index 950d4a7..5944ae2 100644 |--- a/generators/plugin_migration/plugin_migration_generator.rb

    |+++ b/generators/plugin_migration/plugin_migration_generator.rb

    File to patch: ^C Mochi:plugins samuel$

    Am I doing something wrong?

    Kind regards, Samuel

  • Samuel Williams

    Samuel Williams October 6th, 2008 @ 06:46 PM

    Hi Guys,

    I've worked on the patch a bit more. I made the task more robust, so that even in some strange situations it still works. For example, one older test I had, had multiple entires for the same plugin and version. That is now sorted using 'uniq' on the initial select of the old schema info table.

    I've also made the insertion of new version information only occur if it doesn't already exist.

    I've attached the latest patch.

  • Samuel Williams

    Samuel Williams October 8th, 2008 @ 11:22 AM

    Sven, James,

    What needs to be done to get this patch integrated?

    Kind regards, Samuel

  • Samuel Williams

    Samuel Williams October 22nd, 2008 @ 01:55 PM

    Hi Guys,

    I got sick of waiting for things to happen, so I've forked engines. Anyone who is interested in getting stuff working is welcome to use my fork. It is compatible with 2.1.0 and is now distributed as a gem. It has quite a few changes and modifications, so please take the time to test things out before you use in production.


    Kind regards and thanks to everyone for their help. Samuel

  • Samuel Williams

    Samuel Williams October 22nd, 2008 @ 01:57 PM

    Tekin, I'd like to include your name in the credits for the migrations stuff, are you happy with that?

  • Tekin

    Tekin October 22nd, 2008 @ 09:50 PM

    Good work for getting it working as a Gem! I have no problem with a credit. How does it fair with 2.1.1 and edge rails?

  • Samuel Williams

    Samuel Williams October 22nd, 2008 @ 09:55 PM

    I've worked on some basic prerequisites for supporting 2.1.1 but it isn't a goal at the moment. I want to have really solid support for 2.1.0 before I focus on anything else. And the current release is really an alpha - it works fine, but its subject to change while I get the whole engines as gems thing sorted out.

    Kind regards Samuel

  • James Adam

    James Adam October 23rd, 2008 @ 04:46 AM


    I appreciate your frustration about lack of progress here, and I hope you're not too annoyed.

    The reason why I hadn't commit anything to the main repository is partially because I'm quite busy, but mainly because I hadn't yet heard a concensus from the interested parties on a particular approach. I was hoping that everyone would try out each other's implementations, and we'd naturally discover a 'winner', but I never got the impression that this had happened.

    I appreciate your passion, and I think it's great that you've created a fork on github. I hope that people will give feedback!

    I would very much love to pull community-approved fixes and features back into the 'main' repository, I'm really just waiting to hear the feedback.

  • Tekin

    Tekin October 23rd, 2008 @ 07:40 AM

    I'm comfortable with the new migration scheme. It's not doing anything that rails doesn't do already, i.e. store a unique identifier for each applied migration in the schema_migrations table.

    Currently, rails uses each migration's timestamp. With this new implementation, we are simply extend that behaviour by appending the plugin name to the timestamp for plugin migrations.

    We no longer need a separate table for plugin migrations, there's less magic and we follow rails' behaviour more closely. Plus, when 2.2 comes out with the option to use sequential instead of timestamped migrations, in theory, it should just work!

    The upgrade script has behaved itself for my apps and seems solid.

    +1 from me.

  • Samuel Williams

    Samuel Williams October 23rd, 2008 @ 12:13 PM


    I am not frustrated nor am I annoyed. I'm quite happy to work on this project. I think most people are happy with the patch, lots of people have tried it out.

    However, there are many fundamentally broken things about engines.

    This is why I've decided to fix them and release a fork. I can't take several months to patch one particular area when there are many areas that need to be fixed. I wouldn't get anything done and it would be next year before we ended up with a working engines for rails 2.1.0.

    Because of the new gem plugin mechanisms, lots of things in engines are broken. My fork will address all these issues, sometimes in a way that is not backwards compatible. However, I will provide documentation regarding how to upgrade and move forward to those who are interested in using the fork.

    This documentation is available on the wiki http://wiki.oriontransfer.org/?r...:engines

    Thanks for the time to address me personally, and I appreciate everyone is busy. As I have stated previously, I will continue to work on this fork for as long as it is relevant, therefore, please consider it as almost a completely separate project. Even though it has the same roots, I am growing a new tree, and I'm very glad that is possible and I have many thanks to those who make it possible.

    As it is, you are welcome to take whatever changes I have made back into the official engines plugin. However, I will be rapidly enhancing the plugin and moving away from its current direction. This may mean that many patches are not compatible or don't fit with the traditional goals of the official plugin. For example, yesterday I made about 5 major changes. Today, there are a few more. which I am still testing.

    I'm producing a format for creating gems which also work with engines, so that engine plugins no longer need to be limited to the plugins directory. I will be releasing some example engine gems soon, so if you have time please take a look and get a feel for what I am doing and trying to achieve.

    The wiki page is a great place to start if you are interested in what I am doing.

    Kind regards and many thanks to everyone.

  • James Adam

    James Adam October 24th, 2008 @ 02:25 AM

    • Title changed from “migrations do not work beyond the initial migration.” to “Migrations should be consistent with Rails' mechanism”

    I'm renaming this ticket to be more clear about my intentions towards fixing it.

    My sincere apologies to everyone who's been waiting for a version of the engines plugin with 'proper' support for timestamped migrations.

    I've looked through Samuel's fork, and most of the work does seem to be focused around making the plugin into a gem, and to support gem'd plugins. This is definitely great work, and I hope that much of it can be merged back here.

  • Samuel Williams

    Samuel Williams October 24th, 2008 @ 10:19 AM

    The main patch for this functionality is this one:


  • James Adam

    James Adam November 9th, 2008 @ 06:52 AM

    • State changed from “open” to “resolved”

    I have pushed the code from 'use_rails_own_migration_mechanism' into the master branch, and so hopefully that represents the resolution of this ticket!


    I'm going to mark this ticket as resolved - fingers crossed that it is!

  • Tekin

    Tekin November 10th, 2008 @ 09:01 AM

    Great news James!

    You may also want to apply these two patches relating to upgrading from Sven's fork and the generator generating migrations with filenames that are too long. They appear to apply cleanly to the master branch.

    Thanks again.

  • James Adam

    James Adam November 13th, 2008 @ 09:25 AM

    Done - both patches applied, and the edge branch has been rebased.

    Thanks everyone!

    On 10 Nov 2008, at 19:01, Lighthouse wrote:

  • Tekin

    Tekin November 13th, 2008 @ 10:55 AM

    Great, thanks!

    For future reference, if you apply git formatted patches with: git am < patch.diff

    The commit details are retained and the creator gets a credit in the commit history.

  • James Adam

    James Adam November 13th, 2008 @ 12:05 PM

    I used git-apply, but I wouldn't be surprised if I screwed it up a bit :)

    On 13 Nov 2008, at 20:55, Lighthouse wrote:

  • Samuel Williams

    Samuel Williams January 14th, 2009 @ 05:04 PM

    I have to apologize. There is a bug with the migration mechanism.

    Rails gets confused with the number-engine format. What happens is that the migrations "1-my_engine" and "1" are considered the same. The patch changes the format to "my-engine-1".

    This is fairly important.

  • James Adam

    James Adam January 15th, 2009 @ 02:37 AM

    • State changed from “resolved” to “open”

    Thanks for posting to the lighthouse, Samuel.

    The first thing to do is write one or more failing tests, so we have a concrete idea of the bug. If you get a chance to do this, that would be great. Otherwise I will look into it.

    BTW, I noticed your patch doesn't match the current state of the engines master branch - it would be most useful if any test work applied cleanly.

    Thanks again,


Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

The rails engines plugin itself

Shared Ticket Bins

Referenced by