utf8 Namespace Reference

Functions for converting Unicode wide-char strings to UTF-8 encoded strings, back and forth. More...

Classes
class	invalid_utf8_exception
	Thrown by operations encountering invalid UTF-8 data. More...

Typedefs
typedef ucs4::iterator_base< std::string_view, ucs4_convert_impl::convert_impl< char >::type >	iterator

Functions
static int	byte_size_from_utf8_first (const unsigned char ch)
std::string	lowercase (std::string_view s)
	Returns a lowercased version of the string.
std::size_t	index (std::string_view str, const std::size_t index)
	Codepoint index corresponding to the nth character in a UTF-8 string.
std::size_t	size (std::string_view str)
	Length in characters of a UTF-8 string.
std::size_t	size (const std::string::const_iterator &start, const std::string::const_iterator &end)
std::string &	insert (std::string &str, const std::size_t pos, const std::string &insert)
	Insert a UTF-8 string at the specified position.
std::string &	erase (std::string &str, const std::size_t start, const std::size_t len=std::string::npos)
	Erases a portion of a UTF-8 string.
std::string &	truncate (std::string &str, const std::size_t size)
	Truncates a UTF-8 string to the specified number of characters.
void	truncate_as_ucs4 (std::string &str, const std::size_t size)
	Truncates a UTF-8 string to the specified number of characters.

Detailed Description

Functions for converting Unicode wide-char strings to UTF-8 encoded strings, back and forth.

Typedef Documentation

iterator

typedef ucs4::iterator_base<std::string_view, ucs4_convert_impl::convert_impl<char>::type> utf8::iterator

Definition at line 41 of file unicode.hpp.

Function Documentation

byte_size_from_utf8_first()

static int utf8::byte_size_from_utf8_first ( const unsigned char  ch )

static

Definition at line 35 of file unicode.cpp.

References count_leading_ones().

Referenced by index(), size(), and size().

erase()

std::string & utf8::erase	(	std::string &	str,
		const std::size_t	start,
		const std::size_t	len = std::string::npos
	)

Erases a portion of a UTF-8 string.

Parameters

str	UTF-8 encoded string.
start	Start position.
len	Number of characters to erase.

Note

This implementation does not check for valid UTF-8. Don't use it for user input.

Definition at line 105 of file unicode.cpp.

References index(), and size().

Referenced by gui2::combobox::delete_selection(), gui2::multiline_text::delete_selection(), gui2::password_box::delete_selection(), gui2::text_box::delete_selection(), and truncate().

index()

std::size_t utf8::index	(	std::string_view	str,
		const std::size_t	index
	)

Codepoint index corresponding to the nth character in a UTF-8 string.

Exceptions

invalid_utf8_exception

if the string is not a valid UTF-8 string. The checking is partial. No matter what is in between, it will iterate from first byte of a character to the next first byte of the following character.

Returns

str.length() if there are less than index characters.

Definition at line 70 of file unicode.cpp.

References byte_size_from_utf8_first(), i, and index().

Referenced by BOOST_AUTO_TEST_CASE(), gui2::text_box_base::copy_selection(), wfl::builtins::DEFINE_WFL_FUNCTION(), erase(), index(), insert(), and font::pango_text::insert_text().

insert()

std::string & utf8::insert	(	std::string &	str,
		const std::size_t	pos,
		const std::string &	insert
	)

Insert a UTF-8 string at the specified position.

Definition at line 100 of file unicode.cpp.

References index(), and insert().

Referenced by gui2::text_box_base::handle_editing(), insert(), gui2::password_box::insert_char(), and font::pango_text::insert_text().

lowercase()

std::string utf8::lowercase

(

std::string_view

s

)

Returns a lowercased version of the string.

Definition at line 50 of file unicode.cpp.

References ucs4::iterator_base< string_type, update_implementation >::end(), s, and ucs4::iterator_base< string_type, update_implementation >::substr().

Referenced by wesnothd::server::bans_handler(), BOOST_AUTO_TEST_CASE(), wesnothd::server::dul_handler(), campaignd::blacklist::is_in_globlist(), wesnothd::server::is_login_allowed(), filesystem::looks_like_pbl(), wesnothd::ban_manager::parse_time(), wesnothd::server::process_command(), wesnothd::server::searchlog_handler(), wesnothd::server::status_handler(), gui2::dialogs::chat_log::model::stream_log(), and campaignd::server::validate_addon().

size() [1/2]

std::size_t utf8::size	(	const std::string::const_iterator &	start,
		const std::string::const_iterator &	end
	)

Definition at line 90 of file unicode.cpp.

References byte_size_from_utf8_first().

size() [2/2]

std::size_t utf8::size

(

std::string_view

str

)

Length in characters of a UTF-8 string.

Exceptions

invalid_utf8_exception

if the string is not a valid UTF-8 string. The checking is partial. No matter what is in between, it will iterate from first byte of a character to the next first byte of the following character.

Definition at line 81 of file unicode.cpp.

References byte_size_from_utf8_first(), and i.

Referenced by gui2::rich_label::add_link(), BOOST_AUTO_TEST_CASE(), erase(), gui2::text_box_base::get_composition_length(), gui2::rich_label::get_parsed_text(), gui2::text_box_base::handle_editing(), gui2::text_box_base::handle_key_right_arrow(), gui2::text_box_base::insert_char(), gui2::password_box::insert_char(), font::pango_text::insert_text(), hotkey::hotkey_keyboard::matches_helper(), markup::position_info(), gui2::dialogs::file_dialog::set_input_text(), gui2::password_box::set_value(), truncate(), truncate_as_ucs4(), and gui2::multiline_text::update_canvas().

truncate()

std::string & utf8::truncate	(	std::string &	str,
		const std::size_t	size
	)

Truncates a UTF-8 string to the specified number of characters.

Parameters

str	UTF-8 encoded string.
size	Size to truncate to.

Note

This implementation does not check for valid UTF-8. Don't use it for user input.

Definition at line 118 of file unicode.cpp.

References erase(), and size().

Referenced by BOOST_AUTO_TEST_CASE(), utils::ellipsis_truncate(), and font::pango_text::set_maximum_length().

truncate_as_ucs4()

void utf8::truncate_as_ucs4	(	std::string &	str,
		const std::size_t	size
	)

Truncates a UTF-8 string to the specified number of characters.

If the string has more than size UTF-8 characters it will be truncated to this size.

The output is guaranteed to be valid UTF-8.

Parameters

[in,out]	str	[in] String encoded in UTF-8. [out] String encoded UTF-8 that contains at most size codepoints.
	size	The size to truncate to.

Definition at line 123 of file unicode.cpp.

References size().

Referenced by wesnothd::chat_message::truncate_message().