Difference between revisions of "Git asynchronous development"

From relax wiki
Jump to navigation Jump to search
m (Switch to the {{relax developer link}} template to remove dead Gna! links.)
 
(56 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
{{historical svn}}
 +
{{caution|This article is out of date, please see the [[Git_svn|article on using git-svn]] for a more appropriate development strategy.}}
 +
 
== Motivation ==
 
== Motivation ==
 
Subversion needs an online repository, to store each commits. Subsequent calls to '''svn diff > patch''' will generate the difference according to the last revision. Therefore the development at the moment, require to  
 
Subversion needs an online repository, to store each commits. Subsequent calls to '''svn diff > patch''' will generate the difference according to the last revision. Therefore the development at the moment, require to  
Line 15: Line 18:
  
 
== Initialization of git ==
 
== Initialization of git ==
−
Navigate to the root folder of the branch of relax, you want to develop. Initialize with:
+
Navigate to the root folder of the branch of relax, you want to develop. Then follow the this post [[Git_svn#relax_branches_at_github | relax_branches_at_github]]
−
git init
 
  
 
=== Setup commit information ===
 
=== Setup commit information ===
Line 22: Line 24:
  
 
== Example ==
 
== Example ==
 +
=== Preparation ===
 
As an example, we can take the development of the NMRPipe SeriesTab reader: [https://gna.org/support/index.php?3043 sr #3043: Support for NMRPipe seriesTab format *.ser]
 
As an example, we can take the development of the NMRPipe SeriesTab reader: [https://gna.org/support/index.php?3043 sr #3043: Support for NMRPipe seriesTab format *.ser]
  
−
  git add
+
We are going to make a support request '''sr''' branch. [http://git-scm.com/book/en/Git-Branching-Basic-Branching-and-Merging See  ref. on branching]. In this branch we add the files we would like to track for changes. Then we create a branch '''seriestab''' from '''sr'''. We then track and create patches of the differences between '''seriestab''' and '''sr'''.
 +
 
 +
Create the support request '''sr''' branch.
 +
git co -b sr master
 +
# See all branches
 +
git br -a
 +
 
 +
Then we create the '''nmrpipe''' branch (because we edit nmrpipe.py), from where we will create the patch according to the differences of the '''sr''' branch.
 +
git co -b nmrpipe sr
 +
 
 +
=== Checking out a development branch ===
 +
This section is modified from a how-to for [http://codeprairie.net/blogs/chrisortman/archive/2008/01/14/creating-subversion-patches-with-git.aspx Creating subversion patches with git]. We want to be able to create patches and commit messages, which are seamless to patch into the subversion relax repository.
 +
 
 +
=== Modification 1 ===
 +
Then we modify lib/software/nmrpipe.py, and add following
 +
<source lang="python">
 +
# Python module imports.
 +
from re import split
 +
 
 +
# relax module imports.
 +
from lib.errors import RelaxError
 +
</source>
 +
We add the change to be recorded
 +
  git add lib/software/nmrpipe.py
 +
# OR use this, to add changes from all tracked files
 +
git add -u
 +
Then we commit
 +
git commit
 +
commit message. See [[Format_commit_logs]].
 +
<source lang="text">
 +
Imported expected used modules in lib.software.nmrpipe.py.
 +
 
 +
Progress sr #3043: (https://gna.org/support/index.php?3043) Support for NMRPipe seriesTab format *.ser
 +
 
 +
Expected modules for use in lib\software\nmrpipe.py is imported.
 +
 
 +
</source>
 +
To see the commit message
 +
git log
 +
If you would like to replace the last commit info
 +
git commit --amend
 +
Or see last changes by
 +
git log -p
 +
 
 +
Now create your [[Git_patch | git patch]], comparing to the sr branch
 +
git format-patch sr --stdout > nmrpipe.patch
 +
 
 +
=== Modification 2 ===
 +
Then we modify, and add following
 +
<source lang="python">
 +
def read_list_intensity_seriestab(file_data=None, int_col=None):
 +
    """Return the peak intensity information from the NMRPipe SeriesTab peak intensity file.
 +
 
 +
    The residue number, heteronucleus and proton names, and peak intensity will be returned.
 +
 
 +
 
 +
    @keyword file_data: The data extracted from the file converted into a list of lists.
 +
    @type file_data:    list of lists of str
 +
    @keyword int_col:  The column containing the peak intensity data (for a non-standard formatted file).
 +
    @type int_col:      int
 +
    @raises RelaxError: When the expected peak intensity is not a float.
 +
    @return:            The extracted data as a list of lists.  The first dimension corresponds to the spin.  The second dimension consists of the proton name, heteronucleus name, residue number, the intensity value, and the original line of text.
 +
    @rtype:            list of lists of str, str, int, float, str
 +
    """
 +
</source>
 +
We add the change to be recorded
 +
git add lib/software/nmrpipe.py
 +
# OR use this, to add changes from all tracked files
 +
git add -u
 +
Then we commit
 +
git commit
 +
commit message. See [[Format_commit_logs]].
 +
<source lang="text">
 +
Added doc string in lib.software.nmrpipe.py.
 +
 
 +
Progress sr #3043: (https://gna.org/support/index.php?3043) Support for NMRPipe seriesTab format *.ser
 +
 
 +
Doc string and variables for read_list_intensity_seriestab() function in lib.software.nmrpipe.py is provided.
 +
 
 +
</source>
 +
To see the commit message
 +
git log
 +
If you would like to replace the last commit info
 +
git commit --amend
 +
Or see last changes by
 +
git log -p -1
 +
 
 +
Now create your patches, comparing to the sr branch
 +
git format-patch sr --stdout > nmrpipe2.patch
 +
This should provide you with a second patch files, with more commits.
 +
 
 +
=== Getting an overview ===
 +
[http://git-scm.com/book/en/Git-Basics-Viewing-the-Commit-History This section is prepared from this site].
 +
 
 +
Using a GUI to Visualize History. If you like to use a more graphical tool to visualize your commit history, you may want to take a look at a Tcl/Tk program called gitk that is distributed with Git. Gitk is basically a visual git log tool, and it accepts nearly all the filtering options that git log does. Type gitk on the command line in your project.
 +
gitk
 +
 
 +
Let's review how we can change older commit messages, [http://stackoverflow.com/questions/8824971/how-to-git-amend-older-commit how-to amend older commit]. We want to change the commit message we made some commits ago
 +
 
 +
To find out, how long we would like to go back, we use the log
 +
git log
 +
We would like change the commit message with commit hash sha1 of ('''833b91e6064cc8175ad741757ada25edd52c2cce''')
 +
git rebase -i 833b91e6064cc8175ad741757ada25edd52c2cce~1
 +
# Instead of 'sha1' you can use 'HEAD~N', where N is the number of commits before 'HEAD'
 +
git rebase -i HEAD~2
 +
 
 +
Now mark the ones you want to amend with reword '''r''' (replace pick). Now save and exit editor.<br>
 +
In the next editor, you can then write the new message. Save and exit editor.
 +
 
 +
Now create your patch, comparing to the sr branch
 +
git format-patch sr --stdout > nmrpipe3.patch
 +
 
 +
=== Continuation ===
 +
You continue with this cycle.
 +
 
 +
* Modify code
 +
* Track the change
 +
git add -u
 +
* Make a commit message
 +
git commit
 +
 
 +
== Finishing up ==
 +
 
 +
=== Complete test suite ===
 +
First we test if we pass the test we setup for ourself
 +
relax -s Peak_lists.test_read_peak_list_NMRPipe_seriesTab
 +
 
 +
=== Upload patches ===
 +
First delete all the previous made patches.
 +
rm -r *.patch
 +
 
 +
Now create your patches, comparing to the sr branch
 +
git format-patch sr --stdout > nmrpipe.patch
 +
 
 +
Then upload the [[git_patch|git patch]] to the [https://gna.org/support/?group=relax support tracker].
 +
 
 +
== Reviewing suggestions and modify commit ==
 +
There were a number of [http://article.gmane.org/gmane.science.nmr.relax.devel/4120 suggestions] to the patches from {{relax developer link|username=bugman|text=Edward}}.
 +
 
 +
=== Squash commits ===
 +
[http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html You can squash commits], which essentially take smaller commits and combine them into larger ones. This can be useful if you’re are wrapping up the day’s work of code. Then you can also prepare a better commit message.
 +
 
 +
First get an overview, over which commits should be squashed
 +
git log
 +
Let's say you wan't to squash the last 6 commits. Set HEAD~6+1.
 +
git rebase -i HEAD~7
 +
In the pop-up editor, for all the commits who should be squashed, replace '''pick''' with '''s''' or '''squash'''. Do not change the '''pick''' for the first commit.
 +
Save and exit editor.
 +
 
 +
Now compose a new commit message, which can be combined of the earlier commit messages. Comment out lines with "#" which should not be added.
 +
<source lang="text">
 +
Completed NMRPipe SeriesTab reader
 +
 
 +
Progress sr #3043: (https://gna.org/support/index.php?3043) Support for NMRPipe seriesTab format *.ser
 +
 
 +
Completed NMRPipe SeriesTab reader for assignment according to SPARKY format.
 +
Changes implemented according to: http://article.gmane.org/gmane.science.nmr.relax.devel/4120
 +
</source>
 +
Then we test if the changes passed our test
 +
relax -s Peak_lists.test_read_peak_list_NMRPipe_seriesTab
 +
 
 +
=== Changes to pipe_control.spectrum.read() ===
 +
The above test, showed that the reading function had to be adjusted. So now we create a new branch, which is based on the branch we last edited. Meaning we copy the content of the modified files.
 +
git co -b spectrum nmrpipe
 +
git diff --stat --color nmrpipe..spectrum
 +
The last line, should show an empty line for "no changes".
 +
 
 +
Now we change spectrum.py in read(), # NMRPipe SeriesTab, and add a commit message to each commit.
 +
 
 +
=== Finishing up ===
 +
Delete old patches
 +
rm *.patch
 +
 
 +
First change to the first branch we made changes is
 +
git co nmrpipe
 +
git format-patch sr --stdout > nmrpipe.patch
 +
git co spectrum
 +
git format-patch nmrpipe --stdout > spectrum.patch
 +
 
 +
== See also ==
 +
# [http://codeprairie.net/blogs/chrisortman/archive/2008/01/14/creating-subversion-patches-with-git.aspx Recommended!: Creating subversion patches with git]
 +
# [https://gist.github.com/nmacinnis/1387609 Windows and TortoiseSVN-friendly modifications]
 +
# [http://stackoverflow.com/questions/708202/git-format-patch-to-be-svn-compatible git-format-patch-to-be-svn-compatible]
 +
# [http://stackoverflow.com/questions/927358/how-to-undo-the-last-git-commit how-to-undo-the-last-git-commit]
 +
 
 +
 
 +
[[Category:Git]]

Latest revision as of 09:42, 16 October 2020

Caution  The information in this section is out of date but is kept for historical reasons. The relax source code is now hosted in a git rather than svn repository.
Caution  This article is out of date, please see the article on using git-svn for a more appropriate development strategy.

Motivation

Subversion needs an online repository, to store each commits. Subsequent calls to svn diff > patch will generate the difference according to the last revision. Therefore the development at the moment, require to

  1. make some lines of code
  2. make a path file and a commit message
  3. use the support tracker to upload patch and commit message
  4. wait for acceptance
  5. wait for commit to official repository
  6. then do an svn update
  7. then return to point 1

This takes time, and require that repository maintainer is online.
If the above scheme is not followed, the patch files will come out of sync.

This can be solved by using git.

Initialization of git

Navigate to the root folder of the branch of relax, you want to develop. Then follow the this post relax_branches_at_github

Setup commit information

If you havent set the repository variables for author information and commit message, this is good time. See Git_installation.

Example

Preparation

As an example, we can take the development of the NMRPipe SeriesTab reader: sr #3043: Support for NMRPipe seriesTab format *.ser

We are going to make a support request sr branch. See ref. on branching. In this branch we add the files we would like to track for changes. Then we create a branch seriestab from sr. We then track and create patches of the differences between seriestab and sr.

Create the support request sr branch. 

git co -b sr master
# See all branches
git br -a

Then we create the nmrpipe branch (because we edit nmrpipe.py), from where we will create the patch according to the differences of the sr branch. 

git co -b nmrpipe sr

Checking out a development branch

This section is modified from a how-to for Creating subversion patches with git. We want to be able to create patches and commit messages, which are seamless to patch into the subversion relax repository.

Modification 1

Then we modify lib/software/nmrpipe.py, and add following 

# Python module imports.
from re import split

# relax module imports.
from lib.errors import RelaxError

We add the change to be recorded 

git add lib/software/nmrpipe.py
# OR use this, to add changes from all tracked files
git add -u

Then we commit 

git commit

commit message. See Format_commit_logs. 

Imported expected used modules in lib.software.nmrpipe.py.

Progress sr #3043: (https://gna.org/support/index.php?3043) Support for NMRPipe seriesTab format *.ser

Expected modules for use in lib\software\nmrpipe.py is imported.

To see the commit message 

git log

If you would like to replace the last commit info 

git commit --amend

Or see last changes by 

git log -p

Now create your git patch, comparing to the sr branch 

git format-patch sr --stdout > nmrpipe.patch

Modification 2

Then we modify, and add following 

def read_list_intensity_seriestab(file_data=None, int_col=None):
    """Return the peak intensity information from the NMRPipe SeriesTab peak intensity file.

    The residue number, heteronucleus and proton names, and peak intensity will be returned.


    @keyword file_data: The data extracted from the file converted into a list of lists.
    @type file_data:    list of lists of str
    @keyword int_col:   The column containing the peak intensity data (for a non-standard formatted file).
    @type int_col:      int
    @raises RelaxError: When the expected peak intensity is not a float.
    @return:            The extracted data as a list of lists.  The first dimension corresponds to the spin.  The second dimension consists of the proton name, heteronucleus name, residue number, the intensity value, and the original line of text.
    @rtype:             list of lists of str, str, int, float, str
    """

We add the change to be recorded 

git add lib/software/nmrpipe.py
# OR use this, to add changes from all tracked files
git add -u

Then we commit 

git commit

commit message. See Format_commit_logs. 

Added doc string in lib.software.nmrpipe.py.

Progress sr #3043: (https://gna.org/support/index.php?3043) Support for NMRPipe seriesTab format *.ser

Doc string and variables for read_list_intensity_seriestab() function in lib.software.nmrpipe.py is provided.

To see the commit message 

git log

If you would like to replace the last commit info 

git commit --amend

Or see last changes by 

git log -p -1

Now create your patches, comparing to the sr branch 

git format-patch sr --stdout > nmrpipe2.patch

This should provide you with a second patch files, with more commits.

Getting an overview

This section is prepared from this site.

Using a GUI to Visualize History. If you like to use a more graphical tool to visualize your commit history, you may want to take a look at a Tcl/Tk program called gitk that is distributed with Git. Gitk is basically a visual git log tool, and it accepts nearly all the filtering options that git log does. Type gitk on the command line in your project. 

gitk

Let's review how we can change older commit messages, how-to amend older commit. We want to change the commit message we made some commits ago

To find out, how long we would like to go back, we use the log 

git log

We would like change the commit message with commit hash sha1 of (833b91e6064cc8175ad741757ada25edd52c2cce) 

git rebase -i 833b91e6064cc8175ad741757ada25edd52c2cce~1
# Instead of 'sha1' you can use 'HEAD~N', where N is the number of commits before 'HEAD'
git rebase -i HEAD~2

Now mark the ones you want to amend with reword r (replace pick). Now save and exit editor.
In the next editor, you can then write the new message. Save and exit editor.

Now create your patch, comparing to the sr branch 

git format-patch sr --stdout > nmrpipe3.patch

Continuation

You continue with this cycle.

  • Modify code
  • Track the change
git add -u
  • Make a commit message
git commit

Finishing up

Complete test suite

First we test if we pass the test we setup for ourself 

relax -s Peak_lists.test_read_peak_list_NMRPipe_seriesTab

Upload patches

First delete all the previous made patches. 

rm -r *.patch

Now create your patches, comparing to the sr branch 

git format-patch sr --stdout > nmrpipe.patch

Then upload the git patch to the support tracker.

Reviewing suggestions and modify commit

There were a number of suggestions to the patches from Edward.

Squash commits

You can squash commits, which essentially take smaller commits and combine them into larger ones. This can be useful if you’re are wrapping up the day’s work of code. Then you can also prepare a better commit message.

First get an overview, over which commits should be squashed 

git log

Let's say you wan't to squash the last 6 commits. Set HEAD~6+1. 

git rebase -i HEAD~7

In the pop-up editor, for all the commits who should be squashed, replace pick with s or squash. Do not change the pick for the first commit. Save and exit editor.

Now compose a new commit message, which can be combined of the earlier commit messages. Comment out lines with "#" which should not be added. 

Completed NMRPipe SeriesTab reader

Progress sr #3043: (https://gna.org/support/index.php?3043) Support for NMRPipe seriesTab format *.ser

Completed NMRPipe SeriesTab reader for assignment according to SPARKY format.
Changes implemented according to: http://article.gmane.org/gmane.science.nmr.relax.devel/4120

Then we test if the changes passed our test 

relax -s Peak_lists.test_read_peak_list_NMRPipe_seriesTab

Changes to pipe_control.spectrum.read()

The above test, showed that the reading function had to be adjusted. So now we create a new branch, which is based on the branch we last edited. Meaning we copy the content of the modified files. 

git co -b spectrum nmrpipe
git diff --stat --color nmrpipe..spectrum

The last line, should show an empty line for "no changes".

Now we change spectrum.py in read(), # NMRPipe SeriesTab, and add a commit message to each commit.

Finishing up

Delete old patches 

rm *.patch

First change to the first branch we made changes is 

git co nmrpipe
git format-patch sr --stdout > nmrpipe.patch
git co spectrum
git format-patch nmrpipe --stdout > spectrum.patch

See also

  1. Recommended!: Creating subversion patches with git
  2. Windows and TortoiseSVN-friendly modifications
  3. git-format-patch-to-be-svn-compatible
  4. how-to-undo-the-last-git-commit
Retrieved from "http://wiki.nmr-relax.com/wiki/index.php?title=Git_asynchronous_development&oldid=7630"