From cfe2483e13b126c4213d0a4d4759b72778dc5e84 Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Sun, 29 May 2011 17:40:45 -0700 Subject: [PATCH 01/19] Update CHANGELOG for 2.0.0 --- doc/CHANGELOG.rdoc | 70 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 55 insertions(+), 15 deletions(-) diff --git a/doc/CHANGELOG.rdoc b/doc/CHANGELOG.rdoc index 48409363..42add1c4 100644 --- a/doc/CHANGELOG.rdoc +++ b/doc/CHANGELOG.rdoc @@ -2,22 +2,62 @@ == TBD v2.0.0 +* Bug #262: Fix line endings +* Bug #341: Remove English strings from RepositoriesHelper +* Bug #343: Review Gantt and Calender links from 07cf681 +* Bug #345: Entering large numbers for 'Estimated Time' fails with 'Invalid big Decimal Value' +* Bug #346: I18n YAML files not parsable with psych yaml library +* Bug #383: Fix broken tests in unstable +* Bug #389: Context menu doesn't work in Opera +* Bug #390: mysql2 incompatibility in WikiPage model +* Bug #400: Review and fix the Activity event types +* Bug #401: Move JournalsHelpers from aaj to the core +* Bug #403: [AAJ] Attachment has it's files and documents activity provider removed but only documents added +* Bug #404: Move aaj/app/* to core +* Bug #405: Move aaj/test/* to core +* Bug #406: Check for missing Journal code from the AAJ merge +* Bug #407: Add Journal#visible +* Bug #408: Check IssueTest#test_saving_twice_should_not_duplicate_journal_details +* Bug #409: [AAJ] Check that bugfix 784bbccf was merged +* Bug #411: Issue Notes Preview +* Bug #412: Test errors on 1.9.2 after acts_as_journalized merge +* Bug #413: Test errors on 1.8.6 after acts_as_journalized merge +* Bug #414: Remove returning since it causes deprecation warnings +* Bug #415: Wikipages don't store/show the comment correctly +* Bug #419: Issue list context menu not working in IE9 +* Bug #422: cvs test are not working +* Bug #423: Remove explicit render from WikiController#show +* Feature #112: Provide a library function to detect the database type used +* Feature #196: Upgrade to Rails 2.3-latest +* Feature #216: Remove the rubygems hack from boot.rb +* Feature #217: Remove the hack to require a specific i18n version in boot.rb +* Feature #269: Refactor lib/redmine/menu_manager.rb to increase extensibility +* Feature #279: Optional start date on Versions +* Feature #289: Switch to helper :all +* Feature #290: Add bundler +* Feature #310: Option to skip mail notifications on issue updates +* Feature #350: Setting model should use Rails.cache instead of class variable +* Feature #416: Refactor watcher_tag and watcher_link to use css selectors for the replace action +* Task #123: Review and Merge acts_as_journalized +* Task #197: Rake task to manage copyright inside of source files +* Task #288: Review latest Redmine commits +* Task #291: Update documentation to phase out Ruby 1.8.6 * From Redmine v1.1.2 -* Defect #3132: Bulk editing menu non-functional in Opera browser -* Defect #6090: Most binary files become corrupted when downloading from CVS repository browser when Redmine is running on a Windows server -* Defect #7280: Issues subjects wrap in Gantt -* Defect #7288: Non ASCII filename downloaded from repo is broken on Internet Explorer. -* Defect #7317: Gantt tab gives internal error due to nil avatar icon -* Defect #7497: Aptana Studio .project file added to version 1.1.1-stable -* Defect #7611: Workflow summary shows X icon for workflow with exactly 1 status transition -* Defect #7625: Syntax highlighting unavailable from board new topic or topic edit preview -* Defect #7630: Spent time in commits not recognized -* Defect #7656: MySQL SQL Syntax Error when filtering issues by Assignee's Group -* Defect #7718: Minutes logged in commit message are converted to hours -* Defect #7763: Email notification are sent to watchers even if 'No events' setting is chosen -* Feature #7608: Add "retro" gravatars -* Patch #7598: Extensible MailHandler -* Patch #7795: Internal server error at journals#index with custom fields +** Defect #3132: Bulk editing menu non-functional in Opera browser +** Defect #6090: Most binary files become corrupted when downloading from CVS repository browser when Redmine is running on a Windows server +** Defect #7280: Issues subjects wrap in Gantt +** Defect #7288: Non ASCII filename downloaded from repo is broken on Internet Explorer. +** Defect #7317: Gantt tab gives internal error due to nil avatar icon +** Defect #7497: Aptana Studio .project file added to version 1.1.1-stable +** Defect #7611: Workflow summary shows X icon for workflow with exactly 1 status transition +** Defect #7625: Syntax highlighting unavailable from board new topic or topic edit preview +** Defect #7630: Spent time in commits not recognized +** Defect #7656: MySQL SQL Syntax Error when filtering issues by Assignee's Group +** Defect #7718: Minutes logged in commit message are converted to hours +** Defect #7763: Email notification are sent to watchers even if 'No events' setting is chosen +** Feature #7608: Add "retro" gravatars +** Patch #7598: Extensible MailHandler +** Patch #7795: Internal server error at journals#index with custom fields == 2011-05-27 v1.4.0 From e68a869d2d70cf5d9785a2e2957399bbe170d383 Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Sun, 29 May 2011 17:42:49 -0700 Subject: [PATCH 02/19] Update RUNNING_TESTS.rdoc to point to a wiki page --- doc/RUNNING_TESTS.rdoc | 32 ++------------------------------ 1 file changed, 2 insertions(+), 30 deletions(-) diff --git a/doc/RUNNING_TESTS.rdoc b/doc/RUNNING_TESTS.rdoc index c5a52577..b3d6ccf7 100644 --- a/doc/RUNNING_TESTS.rdoc +++ b/doc/RUNNING_TESTS.rdoc @@ -1,31 +1,3 @@ -= Installing gems for testing += Testing ChiliProject -Run `rake gems RAILS_ENV=test` to list the required gems. Run -`rake gems:install RAILS_ENV=test` to install any missing gems. - -== Running Tests - -Run `rake --tasks test` to see available tests. -`rake test` will run the entire testsuite. - -Before running `rake test` you need to configure both development -and test databases. - -== Creating test repositories - -ChiliProject supports a wide array of different version control systems. -To test the support, a test repository needs to be created for each of those. - -Run `rake --tasks test:scm:setup` for a list of available test-repositories or -run `rake test:scm:setup:all` to set up all of them - -== Creating a test ldap database - -ChiliProject supports using LDAP for user authentications. To test LDAP -with ChiliProject, load the LDAP export from test/fixtures/ldap/test-ldap.ldif -into a testing LDAP server. Test that the ldap server can be accessed -at 127.0.0.1 on port 389. - -Setting up the test ldap server is beyond the scope of this documentation. -The OpenLDAP project provides a simple LDAP implementation that should work -good as a test server. +The detailed upgrade instructions are located on the {official website}[https://www.chiliproject.org/projects/chiliproject/wiki/Testing] From 37f6b75749ebd56a693981ef3c9283cf02becf80 Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Sun, 29 May 2011 17:59:51 -0700 Subject: [PATCH 03/19] Added VERSION.special to handle special version releases (RC, distributions, etc) --- lib/redmine/version.rb | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/lib/redmine/version.rb b/lib/redmine/version.rb index 28cc71d5..b0dcb1c7 100644 --- a/lib/redmine/version.rb +++ b/lib/redmine/version.rb @@ -19,6 +19,21 @@ module Redmine MINOR = 4 PATCH = 0 TINY = PATCH # Redmine compat + + # Used by semver to define the special version (if any). + # A special version "satify but have a lower precedence than the associated + # normal version". So 2.0.0RC1 would be part of the 2.0.0 series but + # be considered to be an older version. + # + # 1.4.0 < 2.0.0RC1 < 2.0.0RC2 < 2.0.0 < 2.1.0 + # + # This method may be overridden by third party code to provide vendor or + # distribution specific versions. They may or may not follow semver.org: + # + # 2.0.0debian-2 + def self.special + 'RC1' + end def self.revision revision = nil @@ -48,7 +63,7 @@ module Redmine def self.to_a; ARRAY end def self.to_s; STRING end def self.to_semver - [MAJOR, MINOR, PATCH].join('.') + [MAJOR, MINOR, PATCH].join('.') + special end end end From 790bb362c53633cc1b3faa4e7cb2afea287bed04 Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Mon, 30 May 2011 14:05:40 -0700 Subject: [PATCH 04/19] Fix the merge conflict from 6b8897db5fa5a --- lib/redmine/version.rb | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/lib/redmine/version.rb b/lib/redmine/version.rb index 38e94cd7..5db812c8 100644 --- a/lib/redmine/version.rb +++ b/lib/redmine/version.rb @@ -20,6 +20,21 @@ module Redmine PATCH = 0 TINY = PATCH # Redmine compat + # Used by semver to define the special version (if any). + # A special version "satify but have a lower precedence than the associated + # normal version". So 2.0.0RC1 would be part of the 2.0.0 series but + # be considered to be an older version. + # + # 1.4.0 < 2.0.0RC1 < 2.0.0RC2 < 2.0.0 < 2.1.0 + # + # This method may be overridden by third party code to provide vendor or + # distribution specific versions. They may or may not follow semver.org: + # + # 2.0.0debian-2 + def self.special + 'RC1' + end + def self.revision revision = nil entries_path = "#{RAILS_ROOT}/.svn/entries" From 821d9f5ca4188ae4aab021b6eadb6ac6ea6ab2f5 Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Mon, 30 May 2011 14:20:20 -0700 Subject: [PATCH 05/19] [#437] Move the encoding to the top of pdf.rb for Ruby 1.9 --- lib/redmine/export/pdf.rb | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/redmine/export/pdf.rb b/lib/redmine/export/pdf.rb index 99969725..4e6a3b21 100644 --- a/lib/redmine/export/pdf.rb +++ b/lib/redmine/export/pdf.rb @@ -1,3 +1,4 @@ +# encoding: utf-8 #-- copyright # ChiliProject is a project management system. # @@ -11,8 +12,6 @@ # See doc/COPYRIGHT.rdoc for more details. #++ -# encoding: utf-8 - require 'iconv' require 'rfpdf/fpdf' require 'fpdf/chinese' From a428fa60687a55f2e7d45709c2cf8b7f939c0371 Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Mon, 30 May 2011 14:22:31 -0700 Subject: [PATCH 06/19] Bumping version to 2.0.0RC1 --- lib/redmine/version.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/redmine/version.rb b/lib/redmine/version.rb index 5db812c8..1b4c4673 100644 --- a/lib/redmine/version.rb +++ b/lib/redmine/version.rb @@ -15,8 +15,8 @@ require 'rexml/document' module Redmine module VERSION #:nodoc: - MAJOR = 1 - MINOR = 4 + MAJOR = 2 + MINOR = 0 PATCH = 0 TINY = PATCH # Redmine compat From db3480bdb693e7172c22491592767cadff85c393 Mon Sep 17 00:00:00 2001 From: Holger Just Date: Tue, 31 May 2011 09:59:19 +0200 Subject: [PATCH 07/19] [#436] Fix test relying on trailing whitespace. --- test/unit/repository_test.rb | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/test/unit/repository_test.rb b/test/unit/repository_test.rb index f3f4a3e5..00f3bdc0 100644 --- a/test/unit/repository_test.rb +++ b/test/unit/repository_test.rb @@ -109,17 +109,13 @@ class RepositoryTest < ActiveSupport::TestCase def test_for_changeset_comments_strip repository = Repository::Mercurial.create( :project => Project.find( 4 ), :url => '/foo/bar/baz' ) - comment = <<-COMMENT - This is a loooooooooooooooooooooooooooong comment - - - COMMENT + comment = "This is a looooooooooooooong comment" + (" " * 80 + "\n") * 5 changeset = Changeset.new( :comments => comment, :commit_date => Time.now, :revision => 0, :scmid => 'f39b7922fb3c', :committer => 'foo ', :committed_on => Time.now, :repository => repository ) assert( changeset.save ) assert_not_equal( comment, changeset.comments ) - assert_equal( 'This is a loooooooooooooooooooooooooooong comment', changeset.comments ) + assert_equal( 'This is a looooooooooooooong comment', changeset.comments ) end def test_for_urls_strip From aa07982daa29eaed05f782267c336b6ccb041906 Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Wed, 1 Jun 2011 20:41:33 -0700 Subject: [PATCH 08/19] Fix an edge-edge case bug in journal_attributes_for_user with different users In one case in a plugin (not able to reproduce in core) journal_attributes_for_user would set the :user to something different than what's in updated_by thus assigning the Journal to the wrong user. --- .../lib/redmine/acts/journalized/creation.rb | 4 ++-- .../acts_as_journalized/lib/redmine/acts/journalized/users.rb | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/vendor/plugins/acts_as_journalized/lib/redmine/acts/journalized/creation.rb b/vendor/plugins/acts_as_journalized/lib/redmine/acts/journalized/creation.rb index 031fbcf7..3a5493bc 100644 --- a/vendor/plugins/acts_as_journalized/lib/redmine/acts/journalized/creation.rb +++ b/vendor/plugins/acts_as_journalized/lib/redmine/acts/journalized/creation.rb @@ -120,8 +120,8 @@ module Redmine::Acts::Journalized def journal_attributes attributes = { :journaled_id => self.id, :activity_type => activity_type, :changes => journal_changes, :version => last_version + 1, - :notes => journal_notes, :user_id => (journal_user.try(:id) || User.current) } + :notes => journal_notes, :user_id => (journal_user.try(:id) || User.current.try(:id)) } end end end -end \ No newline at end of file +end diff --git a/vendor/plugins/acts_as_journalized/lib/redmine/acts/journalized/users.rb b/vendor/plugins/acts_as_journalized/lib/redmine/acts/journalized/users.rb index d2b719fd..affa925c 100644 --- a/vendor/plugins/acts_as_journalized/lib/redmine/acts/journalized/users.rb +++ b/vendor/plugins/acts_as_journalized/lib/redmine/acts/journalized/users.rb @@ -57,7 +57,7 @@ module Redmine::Acts::Journalized # Overrides the +journal_attributes+ method to include user information passed into the # parent object, by way of a +updated_by+ attr_accessor. def journal_attributes_with_user - journal_attributes_without_user.merge(:user => updated_by || User.current) + journal_attributes_without_user.merge(:user_id => updated_by.try(:id) || User.current.try(:id)) end end From 76c9a6682f2fbd9d4f4e48a9489739f291eb9f95 Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Fri, 3 Jun 2011 09:15:46 -0700 Subject: [PATCH 09/19] Remove duplicated test --- test/functional/issues_controller_test.rb | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/test/functional/issues_controller_test.rb b/test/functional/issues_controller_test.rb index 18985ea6..39a890a0 100644 --- a/test/functional/issues_controller_test.rb +++ b/test/functional/issues_controller_test.rb @@ -632,17 +632,6 @@ class IssuesControllerTest < ActionController::TestCase assert_equal IssueStatus.default, issue.status end - should "accept default status" do - assert_difference 'Issue.count' do - post :create, :project_id => 1, - :issue => {:tracker_id => 1, - :subject => 'This is an issue', - :status_id => 1} - end - issue = Issue.last(:order => 'id') - assert_equal IssueStatus.default, issue.status - end - should "ignore unauthorized status" do assert_difference 'Issue.count' do post :create, :project_id => 1, From 7eb98dbd341c810e5f25d4efbdc7bfa30ff155f6 Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Fri, 3 Jun 2011 09:55:44 -0700 Subject: [PATCH 10/19] [#443] Fix failing tests on 1.9.2 due to an invalid fixture --- test/fixtures/journals.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/fixtures/journals.yml b/test/fixtures/journals.yml index d738c027..33baef2a 100644 --- a/test/fixtures/journals.yml +++ b/test/fixtures/journals.yml @@ -37,7 +37,7 @@ journals_003: journaled_id: 2 changes: | --- - custom_values_2: + custom_values2: - 'Old value' - 'New value' journals_004: From 81bdbd77f7c976ae6e5b503139e8912694bd944b Mon Sep 17 00:00:00 2001 From: Toshi MARUYAMA Date: Sat, 9 Apr 2011 05:41:12 +0000 Subject: [PATCH 11/19] scm: not use Iconv for log converting in Ruby 1.9 and fix tests fails in Ruby 1.9. git-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@5367 e93f8b46-1217-0410-a6f0-8f06a7374b81 --- app/models/changeset.rb | 25 ++++++++++++++++--------- test/unit/changeset_test.rb | 4 ++-- 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/app/models/changeset.rb b/app/models/changeset.rb index 4fdaf665..6a0f55c1 100644 --- a/app/models/changeset.rb +++ b/app/models/changeset.rb @@ -239,21 +239,28 @@ class Changeset < ActiveRecord::Base private def self.to_utf8(str, encoding) - return str if str.blank? - unless encoding.blank? || encoding == 'UTF-8' - begin - str = Iconv.conv('UTF-8', encoding, str) - rescue Iconv::Failure - # do nothing here - end - end + return str if str.nil? + str.force_encoding("ASCII-8BIT") if str.respond_to?(:force_encoding) + return str if str.empty? + str.force_encoding("UTF-8") if str.respond_to?(:force_encoding) if str.respond_to?(:force_encoding) - str.force_encoding('UTF-8') + enc = encoding.blank? ? "UTF-8" : encoding + if enc != "UTF-8" + str.force_encoding(enc) + str = str.encode("UTF-8") + end if ! str.valid_encoding? str = str.encode("US-ASCII", :invalid => :replace, :undef => :replace, :replace => '?').encode("UTF-8") end else + unless encoding.blank? || encoding == 'UTF-8' + begin + str = Iconv.conv('UTF-8', encoding, str) + rescue Iconv::Failure + # do nothing here + end + end # removes invalid UTF8 sequences begin str = Iconv.conv('UTF-8//IGNORE', 'UTF-8', str + ' ')[0..-3] diff --git a/test/unit/changeset_test.rb b/test/unit/changeset_test.rb index 34520b26..0c2c27c0 100644 --- a/test/unit/changeset_test.rb +++ b/test/unit/changeset_test.rb @@ -258,9 +258,9 @@ class ChangesetTest < ActiveSupport::TestCase def test_comments_should_be_converted_all_latin1_to_utf8 s1 = "\xC2\x80" s2 = "\xc3\x82\xc2\x80" + s4 = s2.dup if s1.respond_to?(:force_encoding) s3 = s1.dup - s4 = s2.dup s1.force_encoding('ASCII-8BIT') s2.force_encoding('ASCII-8BIT') s3.force_encoding('ISO-8859-1') @@ -278,7 +278,7 @@ class ChangesetTest < ActiveSupport::TestCase :scmid => '12345', :comments => s1) assert( c.save ) - assert_equal s2, c.comments + assert_equal s4, c.comments end def test_identifier From 60b34ea29b9687941037b4a3d6d79914a1a0e401 Mon Sep 17 00:00:00 2001 From: Toshi MARUYAMA Date: Sat, 9 Apr 2011 06:34:33 +0000 Subject: [PATCH 12/19] scm: set empty log encoding UTF-8 in Ruby 1.9 and add tests. git-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@5368 e93f8b46-1217-0410-a6f0-8f06a7374b81 --- app/models/changeset.rb | 5 ++++- test/unit/changeset_test.rb | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 1 deletion(-) diff --git a/app/models/changeset.rb b/app/models/changeset.rb index 6a0f55c1..c227b7ec 100644 --- a/app/models/changeset.rb +++ b/app/models/changeset.rb @@ -241,7 +241,10 @@ class Changeset < ActiveRecord::Base def self.to_utf8(str, encoding) return str if str.nil? str.force_encoding("ASCII-8BIT") if str.respond_to?(:force_encoding) - return str if str.empty? + if str.empty? + str.force_encoding("UTF-8") if str.respond_to?(:force_encoding) + return str + end str.force_encoding("UTF-8") if str.respond_to?(:force_encoding) if str.respond_to?(:force_encoding) enc = encoding.blank? ? "UTF-8" : encoding diff --git a/test/unit/changeset_test.rb b/test/unit/changeset_test.rb index 0c2c27c0..1c64d434 100644 --- a/test/unit/changeset_test.rb +++ b/test/unit/changeset_test.rb @@ -281,6 +281,42 @@ class ChangesetTest < ActiveSupport::TestCase assert_equal s4, c.comments end + def test_comments_nil + proj = Project.find(3) + r = Repository::Bazaar.create!( + :project => proj, :url => '/tmp/test/bazaar', + :log_encoding => 'ISO-8859-1' ) + assert r + c = Changeset.new(:repository => r, + :committed_on => Time.now, + :revision => '123', + :scmid => '12345', + :comments => nil) + assert( c.save ) + assert_equal "", c.comments + if c.comments.respond_to?(:force_encoding) + assert_equal "UTF-8", c.comments.encoding.to_s + end + end + + def test_comments_empty + proj = Project.find(3) + r = Repository::Bazaar.create!( + :project => proj, :url => '/tmp/test/bazaar', + :log_encoding => 'ISO-8859-1' ) + assert r + c = Changeset.new(:repository => r, + :committed_on => Time.now, + :revision => '123', + :scmid => '12345', + :comments => "") + assert( c.save ) + assert_equal "", c.comments + if c.comments.respond_to?(:force_encoding) + assert_equal "UTF-8", c.comments.encoding.to_s + end + end + def test_identifier c = Changeset.find_by_revision('1') assert_equal c.revision, c.identifier From 55111556748e2ed3a109f86f911df178ea6f26d9 Mon Sep 17 00:00:00 2001 From: Toshi MARUYAMA Date: Sat, 9 Apr 2011 08:07:22 +0000 Subject: [PATCH 13/19] scm: fix log converting error in Ruby 1.9 and add more tests. git-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@5370 e93f8b46-1217-0410-a6f0-8f06a7374b81 --- app/models/changeset.rb | 3 ++- test/unit/changeset_test.rb | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/app/models/changeset.rb b/app/models/changeset.rb index c227b7ec..9220282d 100644 --- a/app/models/changeset.rb +++ b/app/models/changeset.rb @@ -250,7 +250,8 @@ class Changeset < ActiveRecord::Base enc = encoding.blank? ? "UTF-8" : encoding if enc != "UTF-8" str.force_encoding(enc) - str = str.encode("UTF-8") + str = str.encode("UTF-8", :invalid => :replace, + :undef => :replace, :replace => '?') end if ! str.valid_encoding? str = str.encode("US-ASCII", :invalid => :replace, diff --git a/test/unit/changeset_test.rb b/test/unit/changeset_test.rb index 1c64d434..92a9aeb9 100644 --- a/test/unit/changeset_test.rb +++ b/test/unit/changeset_test.rb @@ -255,6 +255,30 @@ class ChangesetTest < ActiveSupport::TestCase end end + def test_invalid_utf8_sequences_in_comments_should_be_stripped_ja_jis + proj = Project.find(3) + str = "test\xb5\xfetest\xb5\xfe" + if str.respond_to?(:force_encoding) + str.force_encoding('ASCII-8BIT') + end + r = Repository::Bazaar.create!( + :project => proj, + :url => '/tmp/test/bazaar', + :log_encoding => 'ISO-2022-JP' ) + assert r + c = Changeset.new(:repository => r, + :committed_on => Time.now, + :revision => '123', + :scmid => '12345', + :comments => str) + assert( c.save ) + if str.respond_to?(:force_encoding) + assert_equal "test??test??", c.comments + else + assert_equal "testtest", c.comments + end + end + def test_comments_should_be_converted_all_latin1_to_utf8 s1 = "\xC2\x80" s2 = "\xc3\x82\xc2\x80" From 0adf1a87b1a138ae60f58290e1b018a13222210c Mon Sep 17 00:00:00 2001 From: Toshi MARUYAMA Date: Sat, 9 Apr 2011 08:19:55 +0000 Subject: [PATCH 14/19] scm: code clean up app/models/changeset.rb. git-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@5371 e93f8b46-1217-0410-a6f0-8f06a7374b81 --- app/models/changeset.rb | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/app/models/changeset.rb b/app/models/changeset.rb index 9220282d..ad4ac3b2 100644 --- a/app/models/changeset.rb +++ b/app/models/changeset.rb @@ -245,17 +245,18 @@ class Changeset < ActiveRecord::Base str.force_encoding("UTF-8") if str.respond_to?(:force_encoding) return str end - str.force_encoding("UTF-8") if str.respond_to?(:force_encoding) if str.respond_to?(:force_encoding) enc = encoding.blank? ? "UTF-8" : encoding if enc != "UTF-8" str.force_encoding(enc) str = str.encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => '?') - end - if ! str.valid_encoding? - str = str.encode("US-ASCII", :invalid => :replace, - :undef => :replace, :replace => '?').encode("UTF-8") + else + str.force_encoding("UTF-8") + if ! str.valid_encoding? + str = str.encode("US-ASCII", :invalid => :replace, + :undef => :replace, :replace => '?').encode("UTF-8") + end end else unless encoding.blank? || encoding == 'UTF-8' From e19c56d61a73255f648a66156c6b0b70a684c312 Mon Sep 17 00:00:00 2001 From: Toshi MARUYAMA Date: Sat, 9 Apr 2011 09:31:14 +0000 Subject: [PATCH 15/19] scm: replace invalid utf-8 sequences in comments instead of stripping on Ruby 1.8. git-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@5373 e93f8b46-1217-0410-a6f0-8f06a7374b81 --- app/models/changeset.rb | 23 +++++++++++------------ test/unit/changeset_test.rb | 33 ++++++++++++++------------------- 2 files changed, 25 insertions(+), 31 deletions(-) diff --git a/app/models/changeset.rb b/app/models/changeset.rb index ad4ac3b2..bdf25d39 100644 --- a/app/models/changeset.rb +++ b/app/models/changeset.rb @@ -245,8 +245,8 @@ class Changeset < ActiveRecord::Base str.force_encoding("UTF-8") if str.respond_to?(:force_encoding) return str end + enc = encoding.blank? ? "UTF-8" : encoding if str.respond_to?(:force_encoding) - enc = encoding.blank? ? "UTF-8" : encoding if enc != "UTF-8" str.force_encoding(enc) str = str.encode("UTF-8", :invalid => :replace, @@ -259,19 +259,18 @@ class Changeset < ActiveRecord::Base end end else - unless encoding.blank? || encoding == 'UTF-8' - begin - str = Iconv.conv('UTF-8', encoding, str) - rescue Iconv::Failure - # do nothing here - end - end - # removes invalid UTF8 sequences + ic = Iconv.new('UTF-8', enc) + txtar = "" begin - str = Iconv.conv('UTF-8//IGNORE', 'UTF-8', str + ' ')[0..-3] - rescue Iconv::InvalidEncoding - # "UTF-8//IGNORE" is not supported on some OS + txtar += ic.iconv(str) + rescue Iconv::IllegalSequence + txtar += $!.success + str = '?' + $!.failed[1,$!.failed.length] + retry + rescue + txtar += $!.success end + str = txtar end str end diff --git a/test/unit/changeset_test.rb b/test/unit/changeset_test.rb index 92a9aeb9..8496238b 100644 --- a/test/unit/changeset_test.rb +++ b/test/unit/changeset_test.rb @@ -14,7 +14,9 @@ require File.expand_path('../../test_helper', __FILE__) class ChangesetTest < ActiveSupport::TestCase - fixtures :projects, :repositories, :issues, :issue_statuses, :changesets, :changes, :issue_categories, :enumerations, :custom_fields, :custom_values, :users, :members, :member_roles, :trackers + fixtures :projects, :repositories, :issues, :issue_statuses, + :changesets, :changes, :issue_categories, :enumerations, + :custom_fields, :custom_values, :users, :members, :member_roles, :trackers def setup end @@ -235,27 +237,24 @@ class ChangesetTest < ActiveSupport::TestCase assert_equal "Texte encodé en ISO-8859-1.", c.comments end - def test_invalid_utf8_sequences_in_comments_should_be_stripped + def test_invalid_utf8_sequences_in_comments_should_be_replaced_latin1 proj = Project.find(3) str = File.read("#{RAILS_ROOT}/test/fixtures/encoding/iso-8859-1.txt") r = Repository::Bazaar.create!( - :project => proj, :url => '/tmp/test/bazaar', + :project => proj, + :url => '/tmp/test/bazaar', :log_encoding => 'UTF-8' ) assert r - c = Changeset.new(:repository => r, + c = Changeset.new(:repository => r, :committed_on => Time.now, - :revision => '123', - :scmid => '12345', - :comments => str) + :revision => '123', + :scmid => '12345', + :comments => str) assert( c.save ) - if str.respond_to?(:force_encoding) - assert_equal "Texte encod? en ISO-8859-1.", c.comments - else - assert_equal "Texte encod en ISO-8859-1.", c.comments - end + assert_equal "Texte encod? en ISO-8859-1.", c.comments end - def test_invalid_utf8_sequences_in_comments_should_be_stripped_ja_jis + def test_invalid_utf8_sequences_in_comments_should_be_replaced_ja_jis proj = Project.find(3) str = "test\xb5\xfetest\xb5\xfe" if str.respond_to?(:force_encoding) @@ -263,7 +262,7 @@ class ChangesetTest < ActiveSupport::TestCase end r = Repository::Bazaar.create!( :project => proj, - :url => '/tmp/test/bazaar', + :url => '/tmp/test/bazaar', :log_encoding => 'ISO-2022-JP' ) assert r c = Changeset.new(:repository => r, @@ -272,11 +271,7 @@ class ChangesetTest < ActiveSupport::TestCase :scmid => '12345', :comments => str) assert( c.save ) - if str.respond_to?(:force_encoding) - assert_equal "test??test??", c.comments - else - assert_equal "testtest", c.comments - end + assert_equal "test??test??", c.comments end def test_comments_should_be_converted_all_latin1_to_utf8 From 6efba7b05d0438d3b9698316794029f220226ab7 Mon Sep 17 00:00:00 2001 From: Toshi MARUYAMA Date: Sat, 9 Apr 2011 23:21:41 +0000 Subject: [PATCH 16/19] scm: use upcase to compare encoding name "UTF-8" in log converting. git-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@5375 e93f8b46-1217-0410-a6f0-8f06a7374b81 --- app/models/changeset.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/models/changeset.rb b/app/models/changeset.rb index bdf25d39..6804869c 100644 --- a/app/models/changeset.rb +++ b/app/models/changeset.rb @@ -247,7 +247,7 @@ class Changeset < ActiveRecord::Base end enc = encoding.blank? ? "UTF-8" : encoding if str.respond_to?(:force_encoding) - if enc != "UTF-8" + if enc.upcase != "UTF-8" str.force_encoding(enc) str = str.encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => '?') From 0f7ae9b6583ecfc96600dc1497c56323fbdafefc Mon Sep 17 00:00:00 2001 From: Jean-Philippe Lang Date: Sun, 11 Apr 2010 13:55:30 +0000 Subject: [PATCH 17/19] Fixed: Update of Subversion changesets broken by r3466 under Solaris (#5255). git-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@3635 e93f8b46-1217-0410-a6f0-8f06a7374b81 --- app/models/changeset.rb | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/app/models/changeset.rb b/app/models/changeset.rb index 6804869c..971edaf5 100644 --- a/app/models/changeset.rb +++ b/app/models/changeset.rb @@ -272,6 +272,12 @@ class Changeset < ActiveRecord::Base end str = txtar end - str + # removes invalid UTF8 sequences + begin + Iconv.conv('UTF-8//IGNORE', 'UTF-8', str + ' ')[0..-3] + rescue Iconv::InvalidEncoding + # "UTF-8//IGNORE" is not supported on some OS + str + end end end From f518251d5f6a0e53ad6cc4cc188f66487cea4f8e Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Fri, 3 Jun 2011 10:46:46 -0700 Subject: [PATCH 18/19] [#444] Fix Changeset#to_utf8 coding standards --- app/models/changeset.rb | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/app/models/changeset.rb b/app/models/changeset.rb index 971edaf5..bf6f6642 100644 --- a/app/models/changeset.rb +++ b/app/models/changeset.rb @@ -245,21 +245,21 @@ class Changeset < ActiveRecord::Base str.force_encoding("UTF-8") if str.respond_to?(:force_encoding) return str end - enc = encoding.blank? ? "UTF-8" : encoding + normalized_encoding = encoding.blank? ? "UTF-8" : encoding if str.respond_to?(:force_encoding) - if enc.upcase != "UTF-8" - str.force_encoding(enc) + if normalized_encoding.upcase != "UTF-8" + str.force_encoding(normalized_encoding) str = str.encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => '?') else str.force_encoding("UTF-8") - if ! str.valid_encoding? + unless str.valid_encoding? str = str.encode("US-ASCII", :invalid => :replace, :undef => :replace, :replace => '?').encode("UTF-8") end end else - ic = Iconv.new('UTF-8', enc) + ic = Iconv.new('UTF-8', normalized_encoding) txtar = "" begin txtar += ic.iconv(str) From ba3ddee99da5ec49aca90517233d7548947a2ea2 Mon Sep 17 00:00:00 2001 From: Eric Davis Date: Fri, 3 Jun 2011 10:50:16 -0700 Subject: [PATCH 19/19] [#444] Remove extra variable --- app/models/changeset.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/app/models/changeset.rb b/app/models/changeset.rb index bf6f6642..d6ab9142 100644 --- a/app/models/changeset.rb +++ b/app/models/changeset.rb @@ -259,10 +259,10 @@ class Changeset < ActiveRecord::Base end end else - ic = Iconv.new('UTF-8', normalized_encoding) + txtar = "" begin - txtar += ic.iconv(str) + txtar += Iconv.new('UTF-8', normalized_encoding).iconv(str) rescue Iconv::IllegalSequence txtar += $!.success str = '?' + $!.failed[1,$!.failed.length]