The example below features a solution for a common problem. Given that
one knows the internal encoding used by the system for wchar_t
strings, one often is in the position to read text from a file and store
it in wide character buffers. One can do this using mbsrtowcs,
but then we run into the problems discussed above.
int
file2wcs (int fd, const char *charset, wchar_t *outbuf, size_t avail)
{
char inbuf[BUFSIZ];
size_t insize = 0;
char *wrptr = (char *) outbuf;
int result = 0;
iconv_t cd;
cd = iconv_open ("WCHAR_T", charset);
if (cd == (iconv_t) -1)
{
/* Something went wrong. */
if (errno == EINVAL)
error (0, 0, "conversion from '%s' to wchar_t not available",
charset);
else
perror ("iconv_open");
/* Terminate the output string. */
*outbuf = L'\0';
return -1;
}
while (avail > 0)
{
size_t nread;
size_t nconv;
char *inptr = inbuf;
/* Read more input. */
nread = read (fd, inbuf + insize, sizeof (inbuf) - insize);
if (nread == 0)
{
/* When we come here the file is completely read.This still could mean there are some unusedcharacters in the inbuf. Put them back. */
if (lseek (fd, -insize, SEEK_CUR) == -1)
result = -1;
/* Now write out the byte sequence to get into theinitial state if this is necessary. */
iconv (cd, NULL, NULL, &wrptr, &avail);
break;
}
insize += nread;
/* Do the conversion. */
nconv = iconv (cd, &inptr, &insize, &wrptr, &avail);
if (nconv == (size_t) -1)
{
/* Not everything went right. It might only bean unfinished byte sequence at the end of thebuffer. Or it is a real problem. */
if (errno == EINVAL)
/* This is harmless. Simply move the unusedbytes to the beginning of the buffer so thatthey can be used in the next round. */
memmove (inbuf, inptr, insize);
else
{
/* It is a real problem. Maybe we ran out ofspace in the output buffer or we have invalidinput. In any case back the file pointer tothe position of the last processed byte. */
lseek (fd, -insize, SEEK_CUR);
result = -1;
break;
}
}
}
/* Terminate the output string. */
if (avail >= sizeof (wchar_t))
*((wchar_t *) wrptr) = L'\0';
if (iconv_close (cd) != 0)
perror ("iconv_close");
return (wchar_t *) wrptr - outbuf;
}
This example shows the most important aspects of using the iconv
functions. It shows how successive calls to iconv can be used to
convert large amounts of text. The user does not have to care about
stateful encodings as the functions take care of everything.
An interesting point is the case where iconv returns an error and
errno is set to EINVAL. This is not really an error in the
transformation. It can happen whenever the input character set contains
byte sequences of more than one byte for some character and texts are not
processed in one piece. In this case there is a chance that a multibyte
sequence is cut. The caller can then simply read the remainder of the
takes and feed the offending bytes together with new character from the
input to iconv and continue the work. The internal state kept in
the descriptor is not unspecified after such an event as is the
case with the conversion functions from the ISO C standard.
The example also shows the problem of using wide character strings with
iconv. As explained in the description of the iconv
function above, the function always takes a pointer to a char
array and the available space is measured in bytes. In the example, the
output buffer is a wide character buffer; therefore, we use a local
variable wrptr of type char *, which is used in the
iconv calls.
This looks rather innocent but can lead to problems on platforms that
have tight restriction on alignment. Therefore the caller of iconv
has to make sure that the pointers passed are suitable for access of
characters from the appropriate character set. Since, in the
above case, the input parameter to the function is a wchar_t
pointer, this is the case (unless the user violates alignment when
computing the parameter). But in other situations, especially when
writing generic functions where one does not know what type of character
set one uses and, therefore, treats text as a sequence of bytes, it might
become tricky.
Published under the terms of the GNU General Public License