Ramsay Jones [Wed, 13 Jan 2010 20:01:11 +0000 (20:01 +0000)]
win32: Add a 'git__' prefix to the directory reading routines
This reduces the global namespace pollution and allows for
a win32 compiler (eg. Open Watcom) to provide these routines
in a header other than <dirent.h> (eg in <io.h>).
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones [Wed, 19 Aug 2009 16:06:39 +0000 (17:06 +0100)]
Add test-suite coverage testing using gcov
Add a new "coverage" Makefile target that re-builds the
library and tests using the gcc compiler/linker flags
required by gcov, runs the test suite to capture the
runtime data, then compiles a coverage report.
The report, which is saved in a file named "untested",
consists of a list of untested files, followed by a list
of untested functions. More detailed execution statistics
are given in the gcov log files which are saved in the
top-level directory (named like src#hash.c.gcov).
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones [Thu, 27 Aug 2009 15:11:07 +0000 (16:11 +0100)]
Add support for running the tests via valgrind
Add some makefile targets, which use valgrind's memcheck tool to
run the tests, in order to help diagnose memory problems in the
library.
In addition, we enable the '--leak-check' option to report on any
memory leaks. However, unlike the other memory problems reported
by memcheck, memory leak reports do not result in an error exit
from valgrind. (So memory leaks are reported on stderr, but don't
halt the test run.)
A suppressions file (tests.supp) is included since libz triggers
some false positives.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones [Wed, 17 Jun 2009 14:54:59 +0000 (15:54 +0100)]
Disable some msvc "deprecated function" warnings again
In addition to removing the inline #define, commit 209849a also
removed a #pragma to disable msvc deprecated function warnings.
Without this #pragma, msvc currently issues 19 warnings related
to "deprecated insecure c-library functions", such as strcpy()
and 22 warnings related to "deprecated POSIX function names",
such as open().
In order to supress these warnings, re-instate the #pragma.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Add noreturn declaration compatible with the MSVC compiler.
MSVC provides a compiler declaration to declare that a function
never returns. This declaration is required in front of the
function definition rather than at the end, but fortunately gcc
is compatible with this location as well.
Explicit returns are no longer required after calls to test_die.
Signed-off-by: Julio Espinoza-Sokal <julioes@gmail.com> Signed-off-by: Andreas Ericsson <ae@op5.se>
Ramsay Jones [Sun, 14 Jun 2009 21:13:35 +0000 (22:13 +0100)]
win32: Add an fsync() implementation for windows
For information on FlushFileBuffers(), see the msdn document
at msdn.microsoft.com/en-us/library/aa364439(VS.85).aspx
Note that Windows 2000 is shown as the minimum windows version
to support FlushFileBuffers(), so if we wish to support Win9X
and NT4, we will need to add code to dynamically check if
kernel32.dll contains the function.
The only error return mentioned in the msdn document is
ERROR_INVALID_HANDLE, which is returned if the file/device
(eg console) is not buffered. The fsync(2) manpage says that
EINVAL is returned in errno, if "fd is bound to a special
file which does not support synchronization".
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones [Sun, 14 Jun 2009 21:10:23 +0000 (22:10 +0100)]
Correct the spelling of the FLEX_ARRAY macro
When setting the default value, the macro name was specified
as GIT_FLEX_ARRAY, which is inconsistent with it's earlier
usage in the file. This caused a compilation error, using the
MS Visual C/C++ compiler, when compiling the git_packlist
struct definition in src/odb.c.
In addition to changing the spelling of the FLEX_ARRAY macro
to GIT_FLEX_ARRAY, including it's use in src/odb.c, we also
rename the TYPEOF macro to GIT_TYPEOF.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones [Thu, 4 Jun 2009 16:12:59 +0000 (17:12 +0100)]
thread-utils.h: Avoid using a non-standard C construct
In particular, in standard C, a struct or union must have at
least one member declared (ie. structs and unions cannot be
empty). Some compilers allow empty structs as an extension
and won't even issue a warning unless asked for it (eg, gcc
requires -pedantic). Some compilers allow empty structs as
an extension and will only treat it as an error if asked for
strict checking (eg Digital-Mars with -A). Some compilers
simply treat it as an error (eg MS Visual C/C++).
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Andreas Ericsson <ae@op5.se>
Ramsay Jones [Thu, 4 Jun 2009 16:12:00 +0000 (17:12 +0100)]
t0020-dirent.c: Add explicit returns to the callback functions
In particular, the one_entry() and dont_call_me() callback
functions require explicit returns, in order to suppress
some "control path" compiler warnings (from MS Visual C/C++).
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Andreas Ericsson <ae@op5.se>
Ramsay Jones [Thu, 4 Jun 2009 15:44:54 +0000 (16:44 +0100)]
Fix a usage error in a call to the object_file_name() function
In 82324ac, the new static function exists_loose() called
object_file_name() and, in order to detect an error return,
tested for a negative value. This usage is incorrect, as
the error return is indicated by a positive return value.
(A successful call is indicated by a zero return value)
The only error return from object_file_name() relates to
insufficient buffer space and the return value gives the
required minimum buffer size (which will always be >0).
If the caller requires a dynamically allocated buffer,
this allows something like the following call sequence:
size_t len = object_file_name(NULL, 0, db->object_dir, id);
char *buf = git__malloc(len);
if (!buf)
error(...);
object_file_name(buf, len, db->object_dir,id);
...
No current callers take advantage of this capability.
Fix up the call site and change the return type of the
function, from int to size_t, which more accurately
reflects the implementation.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Andreas Ericsson <ae@op5.se>
Ramsay Jones [Mon, 23 Mar 2009 18:47:18 +0000 (18:47 +0000)]
t0020-dirent.c: allow test to be run standalone
This test assumed that it was invoked in an empty directory,
which is true when run from the Makefile, and so would fail
if run standalone. In order to allow the test to work when
run from any directory, create a sub directory "dir-walk"
and chdir() into this directory while running the tests.
Also, add some additional tests.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Fri, 20 Mar 2009 19:51:48 +0000 (19:51 +0000)]
win32: Add routines to abstract memory-mapped file functions
In particular, the git__mmap() and git__munmap() routines provide
the interface to platform specific memory-mapped file facilities.
We provide implementations for unix and win32, which can be found
in their own sub-directories.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Mon, 16 Mar 2009 17:08:45 +0000 (17:08 +0000)]
win32: Open or create files in binary mode
On windows, unless we use the O_BINARY flag in the open()
call, the file I/O routines will perform line ending
conversion (\r\n => \n on input, \n => \r\n on output).
In addition to the performance penalty, most files in the
object database are binary and will, therefore, become
corrupted by this conversion.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Wed, 18 Feb 2009 18:52:13 +0000 (18:52 +0000)]
Fix some (digital-mars) compiler warnings
In particular, conditional expressions which contain an
assignment statement, where the expression type is not
explicitly made to be boolean, elicits the following
message:
warning 2: possible unintended assignment
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Wed, 18 Feb 2009 18:50:36 +0000 (18:50 +0000)]
Avoid using a gcc compiler language extension
In particular, using pointer arithmetic on void pointers,
despite being quite useful, is not legal in standard C.
Avoiding non-standard C constructs will help in porting
the library to other compilers/platforms.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Thu, 26 Feb 2009 20:37:20 +0000 (20:37 +0000)]
Add built-in (Mozilla) SHA1 routines
Paul agreed to the GCC-exception license by email:
|
| From: Paul Kocher <paul@cryptography.com>
| Date: Sun, 15 Mar 2009 11:37:23 -0700
| Subject: Re: Adding Mozilla SHA1 implementation to libgit2
|
| Yes - that's fine.
|
| At 01:56 AM 3/5/2009, Andreas Ericsson wrote:
| > Hi Paul. We spoke earlier about this, if you remember?
| > We'd like to add the GCC-exception to the GPL license
| > for these files.
Signed-off-by: Paul Kocher <paul@cryptography.com> Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Tue, 3 Feb 2009 18:25:13 +0000 (18:25 +0000)]
Rearrange some code to improve clarity
In particular, the test for z-stream input completion
(zs.avail_in != 0) logically belongs with the test for
the Z_STREAM_END stream status. This is also consistent
with the identical check in finish_inflate().
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Tue, 3 Feb 2009 18:23:50 +0000 (18:23 +0000)]
Check for error returns from inflateInit()
At present, it is sufficient to ensure that an error return
from inflateInit() is not ignored. Most error returns, like
Z_VERSION_ERROR and Z_STREAM_ERROR, indicate programming or
build errors. These errors could, perhaps, be handled with
simple asserts. However, for a Z_MEM_ERROR, we may want to
perform some further error handling in the future.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Tue, 3 Feb 2009 18:22:09 +0000 (18:22 +0000)]
Fix a potential memory leak
In particular, neglecting to call inflateEnd() along various
codepaths in the inflate_tail() routine, would result in the
failure to release zlib internal state.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Mon, 12 Jan 2009 19:42:13 +0000 (19:42 +0000)]
Add git__dirname and git__basename utility routines
These routines are intended to extract the directory and
base name from a path string. Note that these routines
do not interact with any filesystem and work only on the
text of the path.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Mon, 12 Jan 2009 19:38:37 +0000 (19:38 +0000)]
Fix a sparse "symbol not declared" warning
In particular, the git__delta_apply() function has not been
declared prior to it's definition. In order to suppress the
warning, include the delta-apply.h header which provides the
public interface. This ensures that the declaration and
definition are consistent.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sat, 3 Jan 2009 12:21:30 +0000 (04:21 -0800)]
Add the binary delta apply algorithm for pack style deltas
The git__delta_apply() function can be used to apply a Git style
delta, such as those used in pack files or in git patch files,
to recover the original object stream.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sat, 3 Jan 2009 11:34:09 +0000 (03:34 -0800)]
Precompute the fanout decoding and the oid offset in a pack-*.idx
The fanout table is fairly commonly accessed, we need to read it
twice for each object we lookup in any given pack file. Most of
the processors running Git are running in little-endian mode, as
they are variants of the x86 platform, so reading the fanout is
a costly operation as we need to convert from network byte order
to local byte order. By decoding the fanout table into a malloc
obtained buffer we can save these 2 decode operations per lookup
and make search go more quickly.
This also cleans up the initialization of the search functions
by cutting out a few instructions, saving a small amount of time.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sat, 3 Jan 2009 10:41:26 +0000 (02:41 -0800)]
Add basic support to read pack-*.idx v1 and v2 files
The index data is mapped into memory and then scanned using a
binary search algorithm to locate the matching entry for the
supplied git_oid. The standard fanout hash trick is applied to
reduce the search space by 8 iterations.
Since the v1 and v2 file formats differ in their search function,
due to the different layouts used for the object records, we use
two different search implementations and a virtual function pointer
to jump to the correct version of code for the current pack index.
The single function jump per-pack should be faster then computing
a branch point inside the inner loop of a common binary search.
To improve concurrency during read operations the pack lock is only
held while verifying the index is actually open, or while opening
the index for the first time. This permits multiple concurrent
readers to scan through the same index.
If an invalid index file is opened we close it and mark the
git_pack's invalid bit to true. The git_pack structure is kept
around in its parent git_packlist, but the invalid bit will cause
all future readers to skip over the pack entirely. Pruning the
invalid entries is relatively unimportant because they shouldn't
be very common, a $GIT_DIRECTORY/objects/pack directory tends to
only have valid pack files.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sat, 3 Jan 2009 08:36:10 +0000 (00:36 -0800)]
Add a simple mmap wrapper for cross-platform mmap usage
Win32 has a variant of mmap that is harder to use than POSIX, but
to run natively and efficiently on Win32 we need some form of it.
gitfo_map_ro() provides a basic mmap function for use in locations
where we need read-only random data access to large ranges of a file,
such as a pack-*.idx.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sat, 3 Jan 2009 08:02:25 +0000 (00:02 -0800)]
Refactor pack memory management and locking to be safer
Using an atomic reference counter is difficult to make
cross-platform, as the reference count implementations
are generally processor specific. Its also hard to do
a proper multi-read/single-write implementation.
We now use a simple mutex around the reference count for the list
of packs. Readers grab the mutex and either build the list, or
increment the existing one's reference count. When the reader is
done with the list, the reference count is decremented. In this way
parallel readers are able to operate on the list without worrying
about it being deallocated out from under them.
Individual pack structures are held by reference counts, but we
only care about the list the pack structure is held in. There is
no need to increment/decrement the pack reference counts as we
scan through them during a read operation, the caller holds the
git_packlist and that is sufficient to hold the packs it references.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Fri, 2 Jan 2009 20:51:47 +0000 (20:51 +0000)]
Fix snprintf compiler warning on cygwin
As far as gcc is concerned, the "z size specifier" is available as
an extension to the language, which is available with or without any
-std= switch. (I think you have to go back to 2.95 for a version
of gcc which doesn't work.) Many other compilers have this as an
extension as well (ie without the equivalent of -std=c99).
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sat, 3 Jan 2009 05:48:40 +0000 (21:48 -0800)]
Change the use of asm/atomic.h to require -DGIT_HAS_ASM_ATOMIC
These headers aren't always available; they typically come from the
Linux kernel, but aren't supposed to be exported into the userspace
/usr/include. Modern kernels won't install these and some distros
rm -rf the directory post kernel header install.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 31 Dec 2008 23:35:36 +0000 (15:35 -0800)]
Rewrite git_foreach_dirent into gitfo_dirent
Our fileops API is currently private. We aren't planning on supplying
a cross-platform file API to applications that link to us. If we did,
we'd probably whole-sale publish fileops, not just the dirent code.
By moving it to be private we can also change the call signature to
permit the buffer to be passed down through the call chain. This is
very helpful when we are doing a recursive scan as we can reuse just
one buffer in all stack frames, reducing the impact the recursion has
on the stack frames in the data cache.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 31 Dec 2008 21:38:47 +0000 (13:38 -0800)]
Run ranlib on libgit2.a after archiving it
Some linkers require ranlib to build a symbol table on the archive
in order to work with it. Most platforms that don't have this
requirement permit ranlib as a noop.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 31 Dec 2008 21:20:21 +0000 (13:20 -0800)]
Add a mutex and atomic counter abstraction and implementations
These abstractions can be used to implement an efficient resource
reference counter and simple mutual exclusion. On pthreads we use
pthread_mutex_t, except when we are also on glibc and can directly
use its asm/atomic.h definitions.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 31 Dec 2008 21:16:31 +0000 (13:16 -0800)]
Cleanup our header inclusion order to ensure pthread.h is early
If we are using threads we need to make sure pthread.h comes
in before just about anything else. Some platforms enable
macros that alter what other headers define.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 31 Dec 2008 19:16:41 +0000 (11:16 -0800)]
Rename the test cases to run in specific orders
This way we can be fairly certain we run tests of lower-level
parts of the library before we run tests of higher-level more
complex parts. If there is any problem in a lower-level part
of the library, the earlier test will identify it and stop,
making it easire to troubleshoot the failure.
A rough naming guide has been added for the test suite to
explain the current category structure.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 31 Dec 2008 07:26:38 +0000 (23:26 -0800)]
Set GIT_EOSERR when the OS errno should be consulted
This error code indicates the OS error code has a better value
describing the last error, as it is likely a network or local
file IO problem identified by a C library function call.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 31 Dec 2008 07:21:36 +0000 (23:21 -0800)]
Wrap malloc and friends and report out of memory as GIT_ENOMEM
We now forbid direct use of malloc, strdup or calloc within the
library and instead use wrapper functions git__malloc, etc. to
invoke the underlying library malloc and set git_errno to a no
memory error code if the allocation fails.
In the future once we have pack objects in memory we are likely
to enhance these routines with garbage collection logic to purge
cached pack data when allocations fail. Because the size of the
function will grow somewhat large, we don't want to mark them for
inline as gcc tends to aggressively inline, creating larger than
expected executables.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 31 Dec 2008 06:29:04 +0000 (22:29 -0800)]
Rename the path of the objects directory to be more specific
We're likely to add additional path data, like the path of the
refs or the path to the config file into the git_odb structure,
as it may grow into the repository wrapper. Changing the name
of the objects directory reference makes it more clear should
we later add something else.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 31 Dec 2008 06:07:56 +0000 (22:07 -0800)]
Fix size_t snprintf warning by using PRIuPTR format macro
This is the correct C99 format code for the size_t type when passed
as an argument to the *printf family. If the platform doesn't
define it, we assume %lu and just cross our fingers that its the
proper setting for a size_t on this system. On most sane platforms,
"unsigned long" is the underlying type of "size_t".
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 31 Dec 2008 05:49:38 +0000 (21:49 -0800)]
Support building on Mac OS X by using pthread_getspecific for TLS
The Mach-O format does not permit gcc to implement the __thread
TLS specification, so we must instead emulate it using a single
int cell allocated from memory and stored inside of the thread
specific data associated with the current pthread.
What makes this tricky is git_errno must be a valid lvalue, so
we really need to return a pointer to the caller and deference it
as part of the git_errno macro.
The GCC-specific __attribute__((constructor)) extension is used
to ensure the pthread_key_t is allocated before any Git functions
are executed in the library, as this is necessary to access our
thread specific storage.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Sat, 27 Dec 2008 18:56:16 +0000 (18:56 +0000)]
Fix a bug in gitfo_read_file()
In particular, when asked to read an empty file, this function
calls malloc() with a zero size allocation request. Standard C
says that the behaviour of malloc() in this case is implementation
defined.
[C99, 7.20.3 says "... If the size of the space requested is zero,
the behavior is implementation-defined: either a null pointer is
returned, or the behavior is as if the size were some nonzero
value, except that the returned pointer shall not be used to
access an object."]
Finesse the issue by over-allocating by one byte. Setting the extra
byte to '\0' may also provide a useful sentinel for text files.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Thu, 18 Dec 2008 22:56:14 +0000 (22:56 +0000)]
Add a file reading routine along with an io buffer type
In particular, the gitfo_read_file() routine can be used to slurp
the complete file contents into an gitfo_buf structure. The buffer
content will be allocated by malloc() and may be released by the
gitfo_free_buf() routine. The io buffer type can be initialised
on the stack with the GITFO_BUF_INIT macro.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Thu, 18 Dec 2008 16:17:05 +0000 (08:17 -0800)]
Run tests in their own subdirectory
This way tests can run in parallel without stepping on each other's
temporary work files. If a test passes the directory is removed
completely; if a test fails only empty directories are removed.
This permits inspection of the failed test's left behind state.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones [Wed, 3 Dec 2008 23:53:55 +0000 (23:53 +0000)]
Add a GIT_PATH_MAX constant
The PATH_MAX symbol is often, but not always, defined
in the <limits.h> header. In particular, on cygwin you
need to include this header to avoid a compilation error.
However, some systems define PATH_MAX to be something as
small as 256, which POSIX is happy to allow, while others
allow much larger values. In general it can vary from
one filesystem to another.
In order to avoid the vagaries of different systems, define
our own symbol.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
projects / libgit2.git / log