A utility for merging configuration / sysconfig files – Final GSoC Report

Source: http://www.openclipart.org

Hello,

last week was very busy for me and also very exciting. The end of Google Summer of Code period overlapped the beginning of a new period of my life. During these days, i moved from Salamis(Greece) to Amsterdam(Netherlands) where i am going to study towards a master degree in Artificial Intelligence. As you may understand there were and still are many things for me to arrange, as a person in a total new place/city/country.

So enough about me, what is up with the project? Well, i am very glad to say that the project “ended” and reached a very satisfactory level, the merging functions are now working and i can say that the application is now in an alpha testing phase. There are of course things that always can be improved and a lot of testing to be done in order for this project to take its place and be useful and ready for use in openSUSE. This is the last GSoC report for this project.

Does the GSoC end means that this project is abandoned? Of course not,  as i have already said this may be  the last GSoC report, but this is definitely not the last report for the project. My intention from the beginning was to join the openSUSE developers society and contribute something back to the community. Therefore, I am planning and actually i am already working in order for the project to reach the next stage.  I’m going to keep posting updates on this web-site so if you are interested in the progress of the project please feel free to visit and check for new blog posts.

There couldn’t be a final GSoC report, without the necessary thanks. First of all i would like to thank Google and openSUSE for giving me this great opportunity to work on such an exciting project. I highly recommend on everyone to apply for the next year GSoC if they are able, it is an amazing experience. Not only you have the chance to learn new things, but you can also meet great people. The kind of people who open source communities are filled with. . So speaking of people I would also like to thank my mentor Michal Hrušecký for his help, I strongly feel like I have gained a new friend after all these months of frequent communication! I would also like to thank Petr Gajdos and of course the Augeas developers for their help and support.

This is not a good-bye blog post, the real project is now beginning )

So as always, till the next blog post,

Christos Bountalis

A utility for merging configuration / sysconfig files – Week 11 Report

Source: http://www.openclipart.org

Hello,

A bit late this week’s report. But not without a reason, the last weeks i have been working hard in order to fulfill the initial goals of this project. After lot’s of coding / compiling / testing this week and of course mind storming i can now share with you very good and exciting news.

What is done this week:

-> aug_process_trees is finally done!! That means we can now proceed to the final goal that of implementing the merging functions.
->moved whole code to Augeas version 0.9. Added necessary code and fixed already existing.
->tree_get_children (fixed)
->tree_compare_children (re worked)
->tree_match_combine (added)
->tree_match_lower_level(added)
->tree_child_sort_label(fixed)
->debug_print_treeArray(added)
->debug_print_treeMatchArray(added)

What is to be done:
-> Finish Merging Functions
-> Create First Beta Packages

That is all in a few lines, as GSoC is getting closer to the end, the time available for completing the project is getting less. So I better get back to coding…

Till next week,
Christos Bountalis

A utility for merging configuration / sysconfig files – Week 10 Report

Source: http://www.openclipart.org

Hello again,

this is my report for the 10th week of GSoC. Unfortunately i wasn’t able to create the beta packages i was hoping. There are still some issues to be resolved within the aug_process_tree that hopefully will be solved this weekend.

This week news:
- During some tests i carried out: small errors and space for small tweaks in code was found. Solved all of errors found.

- Code Improvements:
–>New function: struct tree **tree_get_children(struct tree * node)
–>More checks for null values etc in tree matching comparing etc
–>Better handling of comments and null labeled nodes within the configuration files parsed by augeas. I had an idea this week, after discussing with my mentor Michal, the idea was concerning the sorting of the comments and how not to mess with their order in the file. A solution was found based to the idea,at least temporary, and it is already implemented.

- Code Changes:
–>Changed functions:
—->struct tree** tree_child_sort_label(const augeas *aug,struct tree *node) now accepts the tree parent node as a parameter, changes in the code within
—->struct treeMatch* tree_compare_children(const augeas *aug,struct tree **first,struct tree **second) now accepts arrays of struct trees, changes in the code within
—->aug_process_tree still under constant changes.

The reason for all the above changes,the continuous code improvements and the delay in the completion of the aug_process_tree is based mainly due to the reason that i want to create functions that will be further useful by other developers of Augeas and will not just fulfill merging of sysconfig files but it will be possible to used/extended to other configuration files as well.

What is to be done:
-Finish aug_process_tree
-Beta Packages
-Test merging functions
-Code improvements?!

A utility for merging configuration / sysconfig files – Week 9 Report

Source: http://www.openclipart.org

Hello,

Exciting news for the project this week. After solving the problem with the comments and the tree representation of the sysconfig files in Augeas, there was a big leap in the project’s progress. By having test data and through debugging i was able to complete the match algorithm.
The program is now able to traverse through two trees / files, make the necessary matches between the nodes of these two trees and provide the appropriate information to the later stages. That of merging.

What exactly have been done in the previous week:

- Tree representation of sysconfig files
- Matching between two trees nodes completed
- Sorting of labels (Small Fixes)
- tree_compare function complete
- Tree traversal of aug_process_trees ( revisited)

What is for next week:

- Finish merging functions for aug_process_trees ( I feel optimistic that i will surely complete this next week due to the reason that matching was completed)
- Create first beta packages
- Test Test Test
- Improve code

A utility for merging configuration / sysconfig files – Week 7 Report

Source: http://www.openclipart.org

Hello again,

This is the 7th week report for my GSoC project. During the implementation of the matching procedures, i talked in my last blog post, there were some new developments concerning the project. That made the actual matching procedure halt for a while, because there are no effective way at the moment to test the matching progress in the actual implementation. Where is the problem? The problem that occurred is the handling of the special comments that are used in many of the sysconfig files. These comments may appear next to simple description comments but contain useful information that are then interpreted by the program, such information could be for example the type of a variable etc. So what now? This week I am trying to find an effective way to deal with this problem, the initial idea i have is to modify the sysconfig lens, used by Augeas, in order to represent the sysconfig files in a tree form (with more levels that the current) that will be suitable for the matching/sorting algorithms i have already implemented.

What else?

I got a response for my first patch i submitted in Augeas too, and i have learn many useful things from the tips. So among other things this week i had the chance to rework that patch in order to achieve a better result.

What i need to do this week?

- finish the modification of the sysconfig.aug lens.
- implement a way to effectively deal with the comments (special and simple ones).

Conclusion

During the last weeks I am on the toughest part of my gsoc project. I hope that when i finish this part of the project, there will be a burst of progress for the last parts of the initial schedule.

Till next week,

Christos

A utility for merging configuration / sysconfig files – Week 6 Report

Source: http://www.openclipart.org

Hello,

This is my 6th report concerning the progress of my gsoc project. First of all, I want to apologize because of the 5th report that is missing from last week. Due to some events, for me it was almost impossible to concentrate on work. Even though i did some things, it was not enough to create a report on. The previous week made me get out of schedule at least for a week, i will try to catch up this time by working more on weekends starting from this one. Even though in my initial plan there were about two weeks free time in the schedule that i could use in occasions like this.

Anyway, what’s up with the progress. As I have described in previous post, I am currently working on the matching code / algorithm. Some major improvements took place on the aug_process_tree method, which will be responsible for matching the initial tree with those coming as parameters. The tree traversal algorithm is now working completely, some issues however still exist in the matching of the tree nodes. Hopefully, i will be able to resolve this issues very soon, maybe even in the next couple of days.

Also, some basic drafts of the merging functions were added. Each function will complete the appropriate actions that must be carried , and will also represent each of the merging parameters /merging flags that will be used.

Finally, small changes and improvements took place in few other functions as well in the code. The plan for the next days is to complete the matching of the tree nodes and the merging. At least to a point where more more debugging tests would be able to be carried out.

Till next week,
Christos

A utility for merging configuration / sysconfig files – Week 4 Report

Source: http://www.openclipart.org

Hello,

This is the 4rth week report, a lot of coding this week, you can see some code on my project github under branch working. Yes the code there is still not stable, but I am improving it constantly. Till this moment, i didn’t have any feedback except from my mentor, so if you check it out and have some suggestion or something, please feel free to contact me.

What is new in few words?

There have been improvements in many of the functions i did, several new functions have been added in my augeas branch repo, i try to split things up as much as possible, to help me debug things.

Functions Changed/Added:

aug_find_lense
aug_load_file
tree_child_sort_label
aug_process_tree
int label_compare

merge command in augtool was changed a bit too.

What is next?

The basic steps for tree matching have already been implemented, the goal now is to finish it and actually include the merging functions too. The tree matching is the most complex part as far i have understood in this project. By completing it, the other steps in the initial schedule will be implemented easier.

Till next week,
Christos

A utility for merging configuration / sysconfig files – Week 3 Report

Source: http://www.openclipart.org

Hello,

This my regular weekly post on my GSoC project. Even though this week I had to give my last exams for my bachelor degree and some days of the week I had to spent them on reading, I did some progress on the actual project. First of all, I created my first patch for Augeas, yes this is my first patch not for this particular project, but in general. Every week I found my self, reading learning and carrying out new things, this is the coolest and most exciting thing in the GSoC experience until now.

Back to the actual patch, below you can see the description of the patch:


Subject: [PATCH] Added aug_find_lense function to find and return used for loading a
specific file

Aug_find_lense is a function that accepts a path of a file, and tries to
find the lense that was used to load this file in Augeas. Returns either
a string representing the lense if it is located succesfully either null
if the lense could not be found, or the file is not loaded in augeas.


As i had told you before, i am working on a merge method for augtool as well. This command works like this,

merge destination merging-parameters files-to-be-merged

example:
merge /etc/hosts/ (merging-parameters) /tmp/myfile1 /tmp/myfile2 ….. /tmp/myfile3

Until now I am able to use the aug_find lense to locate the lense used for the destination file. Then i have implemented an extra function that can use the name of the lense to load the files-to-merged to augeas. This permits me to have all these files in tree form used by Augeas. The plan now is to finish the code with aug_process_tree function[1] that will try to match the destination with the files-to-be-merged, and will then call the right function according to merging-parameters if a match is found.

What is next?

By having a merge command in augtool, that can load files in the way I mentioned above, i now have a test basis that will permit me to implement and test the merging functions. This must be done in the next two weeks, so i can be in the initial schedule. I am quite optimistic, things will go as planned. This week among with the coding there has been quite a lot of research on tree pattern matching algoriths, I am grateful that there are always nice people in open source communities that provide help and tips. I am back to coding…

Kind Regards,
Christos

[1] Work in Progress – The aug_process_tree function will be like: int aug_process_trees(struct augeas *aug, const char *dest, void (*process)(struct augeas *aug, const char *first, const char *second, void* data), void* data,const char *sources,…);

A utility for merging configuration / sysconfig files – Week 2 Report

Source: http://www.openclipart.org

This is my second report about my gsoc project, well what is new: This week I occupied my self with augeas, it is the first time I try to use and expand an already existing library and utility. Even thought Augeas is very well documented and the people in their community are very helpful, i had some problems combining my thoughts and ideas with the existing code, because i was trying to find out the best way to avoid code duplication etc. That was at least at the start, now at the end of the week I can say that I am more familiar with the Augeas API and this will certainly help me in the coming weeks.

As long as it concerns the actual merging procedure, i began coding an implementation i have thought. Unfortunately, the approach i used was not sufficient generalized to work with all the configuration files. Therefor i had, to erase most of the coding i did these days. However, from every negative thing comes a positive, in this case, I was able to find a better solution that rise some new problems, however i hope I will be able to find out an answer for them in the following days and implement the right code.

Finally, a merge tool command was added in augtool that is able to use the aug_merge command i added in the library, the particular command will accept a source and a dest tree/path and will then try to combine them according to the flags that i will add as enumeration. The final conclusion I must speed up things a bit cause there are unexpected problems that may occur at the implementation at any time.

Best Regards,
Christos

A utility for merging configuration / sysconfig files – Week 1 Report

Source: http://www.openclipart.org

It’s less than a week, that GSoC 2011 coding period have begun. This is my first report for the project.Every Friday I will make a report in this blog to share with you the progress I am making on the project.So what did I do during these first days?

First of all I created an online repo on GitHub and I forked the Augeas project. That will permit me to use version control of my project “GIT” during the implementation and retain an online repo where I can commit my changes to, while experimenting with the source code. This gave me the opportunity to practice more with git, create branches merge commit changes etc. I have to admit that until now, I didn’t used a version control system much, but i really don’t know why?!?! Really git is awesome and can save the developer valuable and useful time.

Next, I continued with trying out OBS for the first time, the openSUSE Build Service, that just renamed to Open Build Service. Even thought i faced some troubles in the start finding my way around (packaging was a new thing for me, and after using obs I have to say that it is not as hard as I initially believed), with the help of my mentor I think I can understand now better the service and use it. But why use OBS to package so early in the project? Well we thought that by using obs we will be able to create packages (please care they are experimental ) with the changes i make, so other people can try them out if they are interested and maybe help with future debugging.

Finally among my initial experiments with Augeas source code, I have created a branch “helloworld” on my git repo, whether i added a simple hw – Hello world command in the augtool that is contained within Augeas. That command prints as you may have guessed , a hello world message on screen. Then i compiled the project by using the sources of the specific branch, and at the end i used obs to create some packages that can be found on my obs repo .

As I had not any important experience with building software and packages, I occupied myself this week mainly with administration actions. Which however, will help the community and of course everyone interested in my gsoc project, to follow the changes, make suggestions, and contribute with new ideas, but will also help in the latter phase of integrating

Conclusion

The progress of the week 1 in bullets:

• created git repo online through GitHub
• practice with git
• compiling packages from git sources
• Learn the basics about using OBS
• experimenting with augeas code, and creating a small command as a test
• creating openSUSE packages through OBS

The GSoC have just begun, and I have already learned many new and very exciting things. This summer will be very interesting for sure!!! Now i will focus on the actual coding and specifically in the merging procedure that will used in Augeas. Till the next report,

Best Regards,
Christos Bountalis

[1]
http://download.opensuse.org/repositories/home:/mpounta/openSUSE_11.4/

Notice: This is the first time I compile and package a project with obs, therefore the packages inside the repo are highly experimental and only for testing purposes.