Project

General

Profile

Actions
Copy link

Task #2448

closed

Dashboard stats wrong

Added by Florian Effenberger over 6 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Guilhem Moulin
Category:
-
Target version:
Team - Q1/2018
Start date:
Due date:
% Done:

0%

Tags:
URL:

Description

Xisco researched on the Dashboard (thanks!), and it seems there are quite some stats wrong:

  1. Why git://gerrit.libreoffice.org/bibisect-win32-5.2, git://gerrit.libreoffice.org/bibisect-win32-5.1, git://gerrit.libreoffice.org/bibisect-win32-5.0 or git://gerrit.libreoffice.org/bibisect-macosx-64-5.0 are listed in the repositories section?
    This make Norbert Thiebaud the most active committer in the last 7 years, which isn't actually true.
    Link: http://goo.gl/nihJGM
  2. In the last 2 years, 176 commits from László Németh are set as empty commits -> why?
    Link: http://goo.gl/EzjfVj
  3. the number of commits by author in core is quite different comparing the dashboard with git-dm or git shortlog.
    In the dashboard, the top 5 from January 1st 2013, 19:00:00.000 to November 2nd 2017, 12:43:14.014 is ( Link http://goo.gl/hL4KTo ):
    Caolán McNamara 20,075
    Stephan Bergmann 13,119
    Noel Grandin 8,042
    Miklos Vajna 7,536
    Michael Stahl 6,555
    while on git-dm is
    Caolán McNamara 14616 (15.3%)
    Stephan Bergmann 11711 (12.2%)
    Noel Grandin 7687 (8.0%)
    Miklos Vajna 5105 (5.3%)
    Tor Lillqvist 4453 (4.7%)
    and on 'git shortlog -sne --since="01 Jan 2013"' is
    14720 Caolán McNamara <>
    11735 Stephan Bergmann <>
    5635 Noel Grandin <>
    4601 Miklos Vajna <>
    4252 Michael Stahl <>
    which is quite similar to git-dm
    Where all those extra commits come from?

Of course, all 3 points described above affects the total number of commits by affiliation. Anyway, it's important to mention that the commit by affiliation in the Dashboard and in git-dm show different data.

For instance, Noel Grandin has 3 different affiliations in the Dashboard ( Link: http://goo.gl/nvJVAd ) because it counts all affiliation throughout the period of time, while git-dm only show the last affiliation.

Guilhem, is this something you can fix?

Actions
Copy link
 #1

Updated by Guilhem Moulin over 6 years ago

Florian Effenberger wrote:

Xisco researched on the Dashboard (thanks!), and it seems there are quite some stats wrong:

  1. Why git://gerrit.libreoffice.org/bibisect-win32-5.2, git://gerrit.libreoffice.org/bibisect-win32-5.1, git://gerrit.libreoffice.org/bibisect-win32-5.0 or git://gerrit.libreoffice.org/bibisect-macosx-64-5.0 are listed in the repositories section?

I mentioned that last August in the private marketing list (Message-ID: <>) but no one followed up on this so I just left the bibisect repos. I think they were added by the bitergia folks; if there is consensus that they should be removed I can do it, of course.

Guilhem, is this something you can fix?

Checking the other two points now.

Actions
Copy link
 #2

Updated by Florian Effenberger over 6 years ago

I mentioned that last August in the private marketing list (Message-ID:
<
<mailto:>>) but no
one followed up on this so I just left the bibisect repos. I think they
were added by the bitergia folks; if there is consensus that they should
be removed I can do it, of course.

I think so - Xisco, can you confirm?

Actions
Copy link
 #3

Updated by Xisco Fauli Tarazona over 6 years ago

Florian Effenberger wrote:

I mentioned that last August in the private marketing list (Message-ID:
<
<mailto:>>) but no
one followed up on this so I just left the bibisect repos. I think they
were added by the bitergia folks; if there is consensus that they should
be removed I can do it, of course.

I think so - Xisco, can you confirm?

I don't think I've read that email, anyway, I think it's good if we get rid of those bibisect repos in the dashboard

Actions
Copy link
 #4

Updated by Guilhem Moulin over 6 years ago

  • Status changed from New to Feedback

2. In the last 2 years, 176 commits from László Németh are set as empty commits -> why?

That seems to reflect what can be found in the repository once we include all branches. The dashboard reports 256 commits between 2016-01-01 00:00:00 UTC and 2018-01-01 00:00:00 UTC, of which 171 are empty.

$ git log --oneline --since='2016-01-01 T00:00:00Z' --until='2018-01-01 T00:00:00Z' --branches --author='László Németh' | wc -l
258
$ git log --oneline --since='2016-01-01 T00:00:00Z' --until='2018-01-01 T00:00:00Z' --branches --author='László Németh' . | wc -l
87
$ echo $((258 - 87))
171

(I'm not aware of a better way to count empty commits, sorry :-P) Not sure where the 2 extra commits come from, perhaps wrong UTF8 encoding, or author vs. committer. Probably irrelevant anyway.

3. the number of commits by author in core is quite different comparing the dashboard with git-dm or git shortlog.

AFAIK the dashboard counts every single commit (probably excluding merges, didn't check) while git shortlog only counts those that are reachable from HEAD (or the optional revision argument).

In the dashboard, the top 5 from January 1st 2013, 19:00:00.000 to November 2nd 2017, 12:43:14.014 is
Caolán McNamara 20,075

$ git log --oneline --since='January 1st 2013, 19:00:00.000' --until='November 2nd 2017, 12:43:14.014' --branches --author='Caolán McNamara' . | wc -l
20184

Stephan Bergmann 13,119

$ git -C /srv/repositories/core.git log --oneline --since='January 1st 2013, 19:00:00.000' --until='November 2nd 2017, 12:43:14.014' --branches --author='Stephan Bergmann' . | wc -l
13142

Etc. Granted the counts are not exact but it's the same order of magnitude, and the missing handful of commits can be explained like above.

Do we want to count only the commits that are reachable from HEAD? Not sure how to that at the moment, I might have to ask the Bitergia folks.

For instance, Noel Grandin has 3 different affiliations in the Dashboard ( Link: http://goo.gl/nvJVAd ) because it counts all affiliation throughout the period of time, while git-dm only show the last affiliation.

And that's a feature not a bug, isn't it? I don't see why we should forget about devs' former affiliations and only pretend that all their commits are under their current affiliation.

Guilhem, is this something you can fix?

I don't see a problem yet. :-)

Actions
Copy link
 #5

Updated by Guilhem Moulin over 6 years ago

Xisco Fauli Tarazona wrote:

Florian Effenberger wrote:

I mentioned that last August in the private marketing list (Message-ID:
<
<mailto:>>) but no
one followed up on this so I just left the bibisect repos. I think they
were added by the bitergia folks; if there is consensus that they should
be removed I can do it, of course.

I think so - Xisco, can you confirm?

I don't think I've read that email, anyway, I think it's good if we get rid of those bibisect repos in the dashboard

Alright, they're gone now. I couldn't find a way to do it in the dashboard but removing the ES indices did the trick:

curl -XDELETE -sS "http://127.0.0.1:9200/git_170123,git/items/_query" \
     -d '{"query":{"wildcard":{"repo_name":"git://gerrit.libreoffice.org/bibisect-*"}}}'

(Had to install the delete-by-query plugin, though.)

Actions
Copy link
 #6

Updated by Guilhem Moulin about 6 years ago

Should we close this or try to make the dashboard only consider commits that are reachable from master (rather than any branch)?

Actions
Copy link
 #7

Updated by Florian Effenberger about 6 years ago

Should we close this or try to make the dashboard only consider commits
that are reachable from master (rather than any branch)?

Can you ask Xisco about that? No hard feelings here

Actions
Copy link
 #8

Updated by Florian Effenberger about 6 years ago

Florian Effenberger wrote:

Should we close this or try to make the dashboard only consider commits
that are reachable from master (rather than any branch)?

Can you ask Xisco about that? No hard feelings here

Ping? :)

Actions
Copy link
 #9

Updated by Guilhem Moulin about 6 years ago

Florian Effenberger wrote:

Florian Effenberger wrote:

Should we close this or try to make the dashboard only consider commits
that are reachable from master (rather than any branch)?

Can you ask Xisco about that? No hard feelings here

Ping? :)

He's in the Watcher list :-P I also brought that up as a discussion item during the Jan 30 team call, and I don't recall hearing any objection either way (at least there is nothing in the minutes) so the ticket can probably be closed.

Actions
Copy link
 #10

Updated by Xisco Fauli Tarazona about 6 years ago

Hello,
Sorry for my last response.
i don't have hard feelings about it either.
It's ok with me if it's closed.
Regards

Actions
Copy link
 #11

Updated by Guilhem Moulin about 6 years ago

  • Status changed from Feedback to Closed

Xisco Fauli Tarazona wrote:

i don't have hard feelings about it either.
It's ok with me if it's closed.

OK, closing :-)

Actions
Copy link
 #12

Updated by Guilhem Moulin over 5 years ago

FYI, since counting counting all commits (non only those in the master branch, but in any branch and/or reachable by any tag) caused some confusion, for the new dashboard I patched perceval to only count commits in the master branch. Cf. http://vm213.documentfoundation.org/app/kibana#/dashboard/Git .

Actions
Copy link

Also available in: Atom PDF