Skip to content

Conversation

masatake
Copy link
Member

@masatake masatake commented Jun 8, 2025

When releasing, I have to update the NEWS file.
One important topic is how parsers are enhanced, with new kinds, roles, fields, and extras (KRFX).
We have added and updated the 'ctags-lang-*' man pages mainly to track the KRFX.
Updating the man pages only to track them is a boring task.
Instead, making ctags itself report new KRFX.

This change adds version members to KRFX definitions.

Various-- list* commands show the version numbers in the VER column:

$ ./ctags --list-kinds-full=Automake
#LETTER NAME      ENABLED REFONLY NROLES MASTER VER DESCRIPTION
D       data      yes     no      0      NONE     0 datum
L       library   yes     no      0      NONE     0 libraries
M       man       yes     no      0      NONE     0 manuals
P       program   yes     no      0      NONE     0 programs
S       script    yes     no      0      NONE     0 scripts
T       ltlibrary yes     no      0      NONE     0 ltlibraries
c       condition yes     yes     1      NONE     0 conditions
d       directory yes     no      6      NONE     0 directories
p       pseudodir no      yes     6      NONE     1 placeholder for EXTRA_, noinst_, ...
s       subdir    yes     no      0      NONE     0 subdirs

--describe-language option was added.

$ ./ctags --describe-language=RpmMacros          
About RpmMacros language
=======================================================
enabled: yes
version: 0.0

Mappings/patterns
-------------------------------------------------------


Mappings/extensions
-------------------------------------------------------


Aliases
-------------------------------------------------------
#ALIAS

Kinds
-------------------------------------------------------
#LETTER NAME  ENABLED REFONLY NROLES MASTER VER DESCRIPTION
m       macro yes     no      0      NONE     0 macros

Roles
-------------------------------------------------------
#KIND(L/N) NAME ENABLED VER DESCRIPTION

Fields
-------------------------------------------------------
#LETTER NAME ENABLED LANGUAGE JSTYPE FIXED OP VER DESCRIPTION

Extras
-------------------------------------------------------
#LETTER NAME ENABLED LANGUAGE FIXED VER DESCRIPTION

Parameters
-------------------------------------------------------
#NAME DESCRIPTION

Sub parsers stacked on this parser
-------------------------------------------------------
#NAME BASEPARSER DIRECTIONS

Implementation specific status
-------------------------------------------------------
allow null tags: no

TODO:

  • implement --list-languages-full
  • update ctags.1.rst
  • add test cases
  • describe versioning in hacking guide
  • ...

Copy link

codecov bot commented Sep 26, 2025

Codecov Report

❌ Patch coverage is 96.00000% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.92%. Comparing base (e6a4947) to head (e112546).
⚠️ Report is 22 commits behind head on master.

Files with missing lines Patch % Lines
main/kind.c 81.81% 2 Missing ⚠️
main/options.c 97.10% 2 Missing ⚠️
main/parse.c 96.77% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4271      +/-   ##
==========================================
+ Coverage   85.88%   85.92%   +0.03%     
==========================================
  Files         248      248              
  Lines       63602    63717     +115     
==========================================
+ Hits        54624    54746     +122     
+ Misses       8978     8971       -7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

In d469bf9, I forgot to update
versionCurrent and versionAge member of the parser.

(This means the parser versioning of ctags-6.2 is broken.
I must backport this change to ctags-6.2 and release 6.2.1.)

Signed-off-by: Masatake YAMATO <yamato@redhat.com>
@masatake masatake force-pushed the main--show-version-in-list branch from 58c55a8 to afede55 Compare October 13, 2025 09:09
@masatake masatake changed the title [WIP] enhance parser versioning main: enhance parser versioning Oct 13, 2025
@masatake masatake marked this pull request as ready for review October 13, 2025 09:11
@masatake masatake force-pushed the main--show-version-in-list branch from afede55 to f6101db Compare October 13, 2025 09:25
@masatake
Copy link
Member Author

An interesting bug(?):

--- ./Tmain/versioning.d/stderr-expected.txt	2025-10-13 09:27:51.780878500 +0000
	+++ D:/a/ctags/ctags/Tmain/versioning.d/stderr-actual.txt	2025-10-13 09:31:00.757293100 +0000
	@@ -1,8 +1,8 @@
	-ctags: Faile to parse the version number for kind "def": -1
	-kind: 1
	-ctags: Faile to parse the version number for role "expanded": -2
	-role: 1
	-ctags: Faile to parse the version number for field "param": -3
	-field: 1
	-ctags: Faile to parse the version number for extra "fullname": -4
	-extra: 1
	+ctags: Warning: the version number (4294967295) of kind "d,def" of language "TEST" should be less than or equal to the current number (10) of the language
	+kind: 0
	+ctags: Warning: the version number (4294967294) of role "expanded" of language "TEST" should be less than or equal to the current number (10) of the language
	+role: 0
	+ctags: Warning: the version number (4294967293) of extra "param" of language "TEST" should be less than or equal to the current number (10) of the language
	+field: 0
	+ctags: Warning: the version number (4294967292) of extra "fullname" of language "TEST" should be less than or equal to the current number (10) of the language
	+extra: 0

It seems that strtoul accepts "-1" on the platform.

@masatake
Copy link
Member Author

masatake commented Oct 13, 2025

strToULong("-1", 10, ...) returns true.

The behavior is valid as strtoul(1) page explains:


RETURN VALUE
       The  strtoul()  function returns either the result of the conversion or,
       if there was a leading minus sign, the negation of  the  result  of  the
       conversion  represented  as an unsigned value, unless the original (non‐
       negated) value would overflow; in the  latter  case,  strtoul()  returns
       ULONG_MAX  and  sets errno to ERANGE.  Precisely the same holds for str‐
       toull() (with ULLONG_MAX instead of ULONG_MAX).

I have expected the strtou functions rejected negative numbers.
I must verify the code using the functions.

make tutil with the following change demonstrating my misunderstanding:

diff --git a/extra-cmds/utiltest.c b/extra-cmds/utiltest.c
index 76d919139..5bd968a3a 100644
--- a/extra-cmds/utiltest.c
+++ b/extra-cmds/utiltest.c
@@ -277,6 +277,15 @@ static void test_routines_strrstr(void)
    TEST_CHECK(strcmp(strrstr("abcdcdb", "cd"), "cdb") == 0);
 }
 
+static void test_routines_strToU(void)
+{
+   unsigned long ul_tmp;
+   unsigned int ui_tmp;
+
+   TEST_CHECK(strToULong("-1", 10, &ul_tmp) == false);
+   TEST_CHECK(strToUInt("-2", 10, &ui_tmp) == false);
+}
+
 static void test_vstring_ncats(void)
 {
    vString *vstr = vStringNew ();
@@ -351,6 +360,7 @@ TEST_LIST = {
    { "intern",           test_intern           },
    { "numarray",         test_numarray         },
    { "routines/strrstr", test_routines_strrstr },
+   { "routines/strToU", test_routines_strToU },
    { "vstring/ncats",    test_vstring_ncats    },
    { "vstring/truncate_leading", test_vstring_truncate_leading },
    { "vstring/EqC",      test_vstring_eqc },

strToU* functions may accept strings starting with "-".
--langdef=LANG{version=...} flag should not.

Signed-off-by: Masatake YAMATO <yamato@redhat.com>
@masatake masatake force-pushed the main--show-version-in-list branch 2 times, most recently from e5a8aad to dd3355d Compare October 13, 2025 22:04
@masatake masatake requested a review from Copilot October 14, 2025 00:15
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances parser versioning in ctags by adding version tracking capabilities for parser language elements (kinds, roles, fields, and extras). The primary goal is to automatically report new KRFX (Kinds, Roles, Fields, eXtras) instead of manually updating documentation.

Key changes include:

  • Added version members to KRFX definitions across all parser types
  • Introduced a new --describe-language option for comprehensive language parser information
  • Added VER column to various --list-* commands to show version numbers
  • Enhanced optlib2c tool to handle version specifications in optlib files

Reviewed Changes

Copilot reviewed 95 out of 99 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
main/colprint.c Added colprintLineAppendColumnVersion function for version column support
main/colprint_p.h Added header declaration for version column functionality
main/field.c Added version validation and VER column to field listings
main/field.h Added version member to fieldDefinition structure
main/kind.c Added version validation and VER column to kind/role listings
main/kind.h Added version member to roleDefinition and kindDefinition structures
main/options.c Implemented --describe-language option and added new list flag options
main/parse.c Added version flag parsing for kinds, roles, fields, and extras
main/parse_p.h Added new function declarations for flag listing
main/ptag.c Added VER column to pseudo-tag listings
main/ptag_p.h Added version member to ptagDesc structure
main/xtag.c Added version validation and VER column to extra listings
main/xtag.h Added version member to xtagDefinition structure
parsers/*.c Added version numbers to various parser definitions
optlib/*.ctags Added version specifications in optlib configuration files
optlib/*.c Updated generated C files with version information
misc/optlib2c Enhanced to parse and generate version information
man/ctags.1.rst.in Updated documentation for new VER column and --describe-language option
docs/news/HEAD.rst Added release notes for new features

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@masatake masatake force-pushed the main--show-version-in-list branch from dd3355d to d71eacf Compare October 14, 2025 00:49
@masatake masatake requested a review from Copilot October 14, 2025 00:49
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 95 out of 99 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@masatake masatake force-pushed the main--show-version-in-list branch 3 times, most recently from a70633a to cf42e24 Compare October 14, 2025 02:12
@masatake masatake requested a review from Copilot October 14, 2025 03:14
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 98 out of 102 changed files in this pull request and generated 1 comment.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@masatake masatake requested a review from Copilot October 14, 2025 03:38
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 98 out of 102 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

main/options.c:1

  • Corrected spelling of 'acceaptable' to 'acceptable'.
/*

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Suggested by Copilot.
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
* add VER column to --list-kinds-full output, and
* add {version=} flag to --kinddef-<LANG> option.

Signed-off-by: Masatake YAMATO <yamato@redhat.com>
* add VER column to --list-roles output, and
* add {version=} flag to --_roledef-<LANG> option.

Signed-off-by: Masatake YAMATO <yamato@redhat.com>
* add VER column to --list-fields output, and
* add {version=} flag to --_fielddef-<LANG> option.

Signed-off-by: Masatake YAMATO <yamato@redhat.com>
* add VER column to --list-extras output, and
* add {version=} flag to --_extradef-<LANG> option.

Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Add VER column to --list-pseudo-tags output.

Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
…o tags

Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Signed-off-by: Masatake YAMATO <yamato@redhat.com>
@masatake masatake force-pushed the main--show-version-in-list branch from cf42e24 to e112546 Compare October 14, 2025 04:02
@masatake masatake requested a review from Copilot October 14, 2025 04:02
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 99 out of 103 changed files in this pull request and generated no new comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@masatake masatake merged commit e9c5b48 into universal-ctags:master Oct 14, 2025
82 of 83 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant