Skip to content

use sqlite3_column_type() for ColumnTypeScanType() #1327

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

use sqlite3_column_type() for ColumnTypeScanType() #1327

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0a6a5dc
use sqlite3_column_type() for ColumnTypeScanType()
alkemir Mar 17, 2025
27e8f68
add clarifying comment for skipping C.SQLITE_NULL
alkemir Mar 19, 2025
855d426
add guards to ColumnTypeScanType. Return interface{} on error or null…
alkemir Mar 24, 2025
76882fe
dont return nullable since its not null
alkemir Mar 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 49 additions & 20 deletions sqlite3_type.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,27 @@ import (
"strings"
)

const (
SQLITE_INTEGER = iota
SQLITE_TEXT
SQLITE_BLOB
SQLITE_REAL
SQLITE_NUMERIC
SQLITE_TIME
SQLITE_BOOL
SQLITE_NULL
)

var (
TYPE_NULLINT = reflect.TypeOf(sql.NullInt64{})
TYPE_NULLFLOAT = reflect.TypeOf(sql.NullFloat64{})
TYPE_NULLSTRING = reflect.TypeOf(sql.NullString{})
TYPE_RAWBYTES = reflect.TypeOf(sql.RawBytes{})
TYPE_NULLBOOL = reflect.TypeOf(sql.NullBool{})
TYPE_NULLTIME = reflect.TypeOf(sql.NullTime{})
TYPE_ANY = reflect.TypeOf(new(any))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well this is unfortunate - this means we are saying *any instead of any. I know that's the existing behavior, but does that even work?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*any is the expected type, see the Rows.Scan() docs:

// If an argument has type *interface{}, Scan copies the value
// provided by the underlying driver without conversion. When scanning
// from a source value of type []byte to *interface{}, a copy of the
// slice is made and the caller owns the result.

Relevant code:

case nil:
    switch d := dest.(type) {
    case *any:
	if d == nil {
		return errNilPtr
	}
	*d = nil
	return nil

In our case, this works:

	for rr.Next() {
		cc, err := rr.ColumnTypes()
		if err != nil {
			log.Fatal(err)
		}

		pointers := make([]any, 0)
		for _, ct := range cc {
			fmt.Printf("%s ", ct.ScanType())
			pointers = append(pointers, reflect.New(ct.ScanType()).Interface())
		}
		fmt.Println()

		if err := rr.Scan(pointers...); err != nil {
			log.Fatal(err)
		}

		fmt.Println(pointers...)
		for _, p := range pointers {
			if pn, ok := p.(**any); ok {
				fmt.Printf("%t ", *pn == nil)
			}
		}

	}

sql.NullInt64 sql.NullString sql.NullFloat64
&{1 true} &{hello true} &{3.1415 true}
sql.NullInt64 *interface {} *interface {}
&{2 true} 0xc000064090 0xc000064098
true true

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's unrelated. Rows.Scan is talking about its arguments, which by necessity all have to be pointers as they are essentially output parameter. Meanwhile, ColumnTypeScanType is talking about the type to scan into (so not a pointer).

For example, ColumnTypeScanType mentions returning reflect.TypeOf(int64(0)) while Rows.Scan mentions *int64.

You'll also notice that all the other types being returned (e.g., TYPE_NULLSTRING) here are not pointers.

In other words, you are supposed to create a variable of the type from ColumnTypeScanType, and then pass a pointer to it to Rows.Scan.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. I will return interface{} now, following Go 1.21 , although this has been changed for Go 1.24 for compatibility.

Still, this might break code depending on the previous behavior.

)

// ColumnTypeDatabaseTypeName implement RowsColumnTypeDatabaseTypeName.
func (rc *SQLiteRows) ColumnTypeDatabaseTypeName(i int) string {
return C.GoString(C.sqlite3_column_decltype(rc.s.s, C.int(i)))
Expand All @@ -39,42 +60,50 @@ func (rc *SQLiteRows) ColumnTypeNullable(i int) (nullable, ok bool) {
}

// ColumnTypeScanType implement RowsColumnTypeScanType.
// In SQLite3, this method should be called after Next() has been called, as sqlite3_column_type()
// returns the column type for a specific row. If Next() has not been called, fallback to
// sqlite3_column_decltype()
func (rc *SQLiteRows) ColumnTypeScanType(i int) reflect.Type {
//ct := C.sqlite3_column_type(rc.s.s, C.int(i)) // Always returns 5
switch C.sqlite3_column_type(rc.s.s, C.int(i)) {
case C.SQLITE_INTEGER:
return TYPE_NULLINT
case C.SQLITE_FLOAT:
return TYPE_NULLFLOAT
case C.SQLITE_TEXT:
return TYPE_NULLSTRING
case C.SQLITE_BLOB:
return TYPE_RAWBYTES
// This case can signal that the value is NULL or that Next() has not been called yet.
// Skip it and return the fallback behaviour as a best effort. This is safe as all types
// returned are Nullable or any, which is the expected value for SQLite3.
//case C.SQLITE_NULL:
// return TYPE_ANY
}

// Fallback to schema declared to remain retro-compatible
return scanType(C.GoString(C.sqlite3_column_decltype(rc.s.s, C.int(i))))
}

const (
SQLITE_INTEGER = iota
SQLITE_TEXT
SQLITE_BLOB
SQLITE_REAL
SQLITE_NUMERIC
SQLITE_TIME
SQLITE_BOOL
SQLITE_NULL
)

func scanType(cdt string) reflect.Type {
t := strings.ToUpper(cdt)
i := databaseTypeConvSqlite(t)
switch i {
case SQLITE_INTEGER:
return reflect.TypeOf(sql.NullInt64{})
return TYPE_NULLINT
case SQLITE_TEXT:
return reflect.TypeOf(sql.NullString{})
return TYPE_NULLSTRING
case SQLITE_BLOB:
return reflect.TypeOf(sql.RawBytes{})
return TYPE_RAWBYTES
case SQLITE_REAL:
return reflect.TypeOf(sql.NullFloat64{})
return TYPE_NULLFLOAT
case SQLITE_NUMERIC:
return reflect.TypeOf(sql.NullFloat64{})
return TYPE_NULLFLOAT
case SQLITE_BOOL:
return reflect.TypeOf(sql.NullBool{})
return TYPE_NULLBOOL
case SQLITE_TIME:
return reflect.TypeOf(sql.NullTime{})
return TYPE_NULLTIME
}
return reflect.TypeOf(new(any))
return TYPE_ANY
}

func databaseTypeConvSqlite(t string) int {
Expand Down